I've seen that there is some information in the first volume of Polya's "Mathematics and Plausible Reasoning". Also, the following paper deals with the usage in probability - "The historical development of the use of generating functions in probability theory" by Seal (https://www.e-periodica.ch/cntmng?pid=msa-001:1949:49::299).
However, I am interested to see whether anybody know any other place where one may find information on the topic. Is there any other literature on the topic?
Seal covers the probabilistic side pretty well. A more recent source with a lot of material on generating functions is Fisher's History of the Central Limit Theorem.
For the combinatorial side, Stanley's Enumerative Combinatorics, v1 has extensive historical notes at the end of each chapter, chapters 4 and 5 are on rational generating functions. The notes to them begin as follows:
"The basic theory of rational generating functions in one variable belongs to the calculus of finite differences. Charles Jordan [4.25] ascribes the origin of this calculus to Brook Taylor in 1717 but states that the real founder was James Stirling in 1730. The first treatise on the subject was written by Euler in 1755, where the notation $\Delta$ for the difference operator was introduced.
The compositional formula (Theorem 5.1.4) and the exponential formula (Corollary 5.1.6) had many precursors before blossoming into their present form. A purely formal formula for the coefficients of the composition of two exponential generating functions goes back to Faa di Bruno [23][24] in 1855 and 1857, and is known as Faa di Bruno's formula. For additional references on this formula, see [2.3, p. 137]. An early precursor of the exponential formula is due to Jacobi [38]. The idea of interpreting the coefficients of $e^{F(x)}$ combinatorially was considered in certain special cases by Touchard [ 69] and by Riddell and Uhlenbeck [56]... It was not until the early 1970s that a general combinatorial interpretation of $e^{F(x)}$ was developed independently by Foata and Schiltzenberger [26], Bender and Goldman [3.3], and Doubilet, Rota, and Stanley [3.12]."
Combinatorial Enumeration by Goulden and Jackson also has notes and references at the end of chapters, but far less extensive than Stanley's.
An excellent source on the history of generating functions (in the broader context of Harmonic Analysis) are the papers of George Mackey:
Harmonic analysis as the exploitation of symmetry - a historical survey. Zbl 0437.43001 Bull. Am. Math. Soc., New Ser. 3, 543-698 (1980).
The scope and history of commutative and noncommutative harmonic analysis. Zbl 0766.43001
