I like to think of L'Hospital as one of the earliest authors of least-squares regression.
L'Hospital, G. (1696). L'analyse des infiniment petits pour l'intelligence des lignes courbes.
I'm sure there are other historic examples, but this is the oldest that I know.
Along the lines of the first publisher of the idea, I was wondering:
Q: What is the oldest example of using regularization? Who used the term or discovered the idea first?
Ridge regression, a regularized regression, has been around quite a while.
Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67.
There are articles on the general topic from the 1930's, but who was the first, the pioneer?