I like to think of L'Hospital as one of the earliest authors of least-squares regression.

L'Hospital, G. (1696). L'analyse des infiniment petits pour l'intelligence des lignes courbes.

I'm sure there are other historic examples, but this is the oldest that I know.

Along the lines of the first publisher of the idea, I was wondering:

Q: What is the oldest example of using regularization? Who used the term or discovered the idea first?

Ridge regression, a regularized regression, has been around quite a while.

Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67.

There are articles on the general topic from the 1930's, but who was the first, the pioneer?

  • 2
    $\begingroup$ "The need to tame, or at least mitigate, these instabilities led Phillips and Tikhonov to independently develop the first regularization methods for approximation of solutions in 1962 and 1963, respectively" C W Groetsch 2007 J. Phys.: Conf. Ser. 73 012001 $\endgroup$
    – ACR
    Commented Apr 19, 2022 at 19:53


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.