6
$\begingroup$

When reading the section related to Gram-Schmidt process in the book Linear Algebra and Its Applications by Gilbert Strang, I found a foot note that says:

If Gram thought of it first, what was left for Schmidt.

Other books taught me that the two independently developed the method and many Internet sources also say the same thing (none go into the details of the contribution of each to the procedure). Thus, I suspect that prof. Strang seems to imply that Gram made the first contribution to the procedure but it was still incomplete and Schmidt later on handled the missing part (probably the part to normalize the basis vectors).

So what exactly did prof. Strang mean by adding the footnote?

$\endgroup$
2
  • 3
    $\begingroup$ FWIW, various commentators opine that the procedure was actually invented by Laplace and already used by Cauchy. J. P. Gram, "Ueber die Entwicklung reeller Functionen in Reihen mittelst der Methode der kleinsten Quadrate." Journal für die reine und angewandte Mathematik, Vol. 94, No. 1, 1883, pp. 41-73. This publication is based in parts on Gram's 1879 Ph.D. thesis (in Danish) at the University of Copenhagen. $\endgroup$
    – njuffa
    Dec 6, 2023 at 8:58
  • 3
    $\begingroup$ Erhard Schmidt, "Zur Theorie der linearen und nichtlinearen Integralgleichungen. I. Teil: Entwicklung willkürlicher Functionen nach System vorgeschriebener." Mathematische Annalen, Vol. 63, No. 4, Mar. 1907, pp. 433-476. Most of the material in this publication originates in Schmidt's 1905 Ph.D. dissertation at the University of Göttingen. Of particular interest appears to be "§ 3. Ersetzung linear unabhängiger Funktionensysteme durch orthogonale" starting on p. 442. $\endgroup$
    – njuffa
    Dec 6, 2023 at 9:00

1 Answer 1

14
$\begingroup$

A detailed history can be found in Gram-Schmidt orthogonalization: 100 years and more by Leon, Björck and Gander, see also their slides for a brief version. In short, Schmidt's 1907 presentation was more explicit and indicated a key property of the process that Gram left out in 1879/1883. It led to the process becoming popular. But the idea predates both of them and can be traced to the third edition of Laplace's Théorie Analytique des Probabilités (1820), where orthogonalization was applied in the course of solving least squares systems to estimate the masses of Jupiter and Saturn.

Here are the highlights:

"In 1907, Erhard Schmidt published a paper on integral equations. In that paper, he made use of an orthonormalization technique that has since become known as the CGS. He remarked in the paper that in essence the formulas were presented by J. P. Gram. The algorithm has become well known and is included as a standard topic in elementary linear algebra textbooks. The earliest linkage of the two names Gram and Schmidt to describe the orthonormalization process appears to be in a 1935 paper by Wong. An apparently slight change in the CGS process gives the modified Gram–Schmidt process (MGS)... This tool, applied to a least squares problem, can be found in a book by Laplace on the analytic theory of probabilities.

J. P. Gram’s original 1879 thesis on integral equations was written in Danish. The results became more accessible to the general mathematics community when a German version was published in 1883. In this paper, Gram was concerned with series expansions of real functions using least squares. Gram was influenced by the work of Chebyshev, and his original orthogonalization procedure was applied to orthogonal polynomials... In the paper, Schmidt applied an orthogonalization process to a sequence of functions $\phi_1\dots\phi_n$ with respect to the inner product $\langle f,g\rangle=\int_a^bf(x)g(x)\,dx$.

The Schmidt paper popularized the orthogonalization process, and as a result of this paper the names Gram and Schmidt have become forever linked. The Schmidt paper did not use inner-product or norm notation... The Gram–Schmidt process has two characteristic properties that are clearly evident in the algorithm presented in Schmidt’s 1907 paper. The first property is that the $k$-th vector $q_k$ (or function) is a linear combination of $a_k$ and its predecessors. Second, the combination is expressed in terms of the vector $a_k$ and the previous GS vectors $q_1,\dots,q_k$. On the other hand, the Gram paper honors the first property but not the second.

In 1816, Laplace published his major treatise Théorie Analytique des Probabilités (Analytic Theory of Probability). The third edition, which appeared in 1820, contains three supplements. In the first supplement, the goal of Laplace is to compute the mass of Jupiter (and Saturn) from a system of normal equations provided by the French astronomer Bouvard and from this same system to compute the distribution of error in the solution assuming a normal distribution of the noise on the observations... The method which Laplace introduces consists in successively projecting the system of equations orthogonally to a column of the matrix $A$. These actions eliminate the associated couple observation/variable from the updated system. Ultimately, Laplace eliminates all the variables but the one of interest in the linear least squares problem, which eliminates all the columns but one in $A$. Laplace is indeed introducing the main technique behind the Gram–Schmidt algorithm (successive orthogonal projections.)"

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.