I remember that I read somewhere that Lorentz remained an ether advocate till his death despite the empirical successes of Einstein's relativity in rejecting any kind of ether. However, I cannot remember the source. Can anybody please give me a possible reference regarding this?
Lorentz did remain sympathetic to ether to the end. However, it was not necessarily to his original conception of ether, and it was not "despite the empirical successes" of special relativity. Special relativity and Lorentz's ether theory not only produce identical empirical predictions but are equivalent in an even stronger sense, see e.g. Szabo. So no empirical successes could distinguish between them, a fact recognized by both Einstein and Lorentz. Their diverging decisions were made on conceptual grounds. And Einstein showed similar ether loss remorse in his views after general relativity, so they were not that far apart.
For Lorentz's evolution on the subject see Goldberg, The Lorentz Theory of Electrons and Einstein's Theory of Relativity and Kox, Hendrik Antoon Lorentz, the ether, and the general theory of relativity, where we read:
"It is well known that until his death in 1928 LORENTZ kept insisting on the usefulness of an ether. In spite of his often-expressed admiration for EINSTEIN'S special theory of relativity, he preferred his own ether-based 'theory of electrons'. LORENTZ admitted that his theory and the special theory of relativity had the same empirical consequences and that the ether could not be experimentally detected, but he maintained that some kind of ether was needed as carrier of the electromagnetic field."
Lorentz's position shows considerable nuance and awareness of factors in theory choice that go beyond empirical consequences. In his Lectures on Theory of Electrons (1909) he admitted intuitive attachment, for example:"I cannot but regard the ether, which can be the seat of an electromagnetic field with its energy and its vibrations, as endowed with a certain degree of substantiality, however different it may be from all ordinary matter." In Lectures on Theoretical Physics (1915) he suggested that the choice was, in part, a matter of taste, and that some ether substitute was needed either way, an opinion shared by Einstein (see below):
"This, however, is a question of taste and of words. For whether there is an ether or not, electromagnetic fields certainly exist, and so also does the energy of the electrical oscillations. If we do not like the name of "ether" we must use another word as a peg to hang all of these things upon. It is not certain whether "space" can be so extended as to take care not only of the geometrical properties but of the electrical ones."
And in Problems of Modern Physics (1922) Lorentz treated it as a "grand scheme of things" choice, and even suggested that Einstein's decision to dispense with the electromagnetic ether was right for Einstein:
"All our theories help us to form pictures or images of the world around us, and we try to do this in such a way that the phenomena may be coordinated as well as possible, and that we may see clearly the way in which they are connected... My notion of time is so definite that I clearly distinguish in my picture what is simultaneous and what is not... As to the ether... though the conception of it has certain advantages, it must be admitted that if Einstein had maintained it he certainly would not have given us his theory, and so we are grateful to him for not having gone along old fashioned roads."
Einstein explained his differences with Lorentz, as well their points of convergence, in his May 1920 address Ether and Relativity (published in 1922) at Lorentz's University of Leiden:
"As to the mechanical nature of the Lorentzian ether, it may be said of it, in a somewhat playful spirit, that immobility is the only mechanical property of which it has not been deprived by H A Lorentz. It may be added that the whole change in the conception of the ether which the special theory of relativity brought about, consisted in taking away from the ether its last mechanical quality, namely, its immobility... More careful reflection teaches us however, that the special theory of relativity does not compel us to deny ether. We may assume the existence of an ether; only we must give up ascribing a definite state of motion to it, i.e. we must by abstraction take from it the last mechanical characteristic which Lorentz had still left it.
[...] Recapitulating, we may say that according to the general theory of relativity space is endowed with physical qualities; in this sense, therefore, there exists an ether. According to the general theory of relativity space without ether is unthinkable; for in such space there not only would be no propagation of light, but also no possibility of existence for standards of space and time (measuring-rods and clocks), nor therefore any space-time intervals in the physical sense. But this ether may not be thought of as endowed with the quality characteristic of ponderable media, as consisting of parts which may be tracked through time. The idea of motion may not be applied to it..
[...] As to the part which the new ether is to play in the physics of the future we are not yet clear. We know that it determines the metrical relations in the space-time continuum, e.g. the configurative possibilities of solid bodies as well as the gravitational fields; but we do not know whether it has an essential share in the structure of the electrical elementary particles constituting matter. Nor do we know whether it is only in the proximity of ponderable masses that its structure differs essentially from that of the Lorentzian ether; whether the geometry of spaces of cosmic extent is approximately Euclidean."