The idea of a hypothetical planet 'Vulcan' with an orbit closer to the Sun than Mercury's orbit has been given excessive prominence in secondary and popular accounts. These accounts tend to produce a distorted narrative of what took place in the astronomical debates over the problem of apparent excess in the motion of Mercury's perihelion.
U J J Le Verrier first reported the Mercury perihelion problem to the Paris Academy of Sciences in 1859 (ref.1), recommending the attention of other astronomers to this serious astronomical problem. At that time, he indicated that potential explanation by an intra-mercurial planet had difficulties, and was an unlikely solution to the problem (although he was soon afterwards induced to change his mind for a time, see below).
In the same paper and at the same time he also discussed classes of conceivable explanation. The possibilities that he considered all involved extra gravitational mass present in the solar system but not yet accounted for.
One class of conceivable explanation would put the extra unaccounted mass somewhere farther out from the Sun than Mercury's orbit. Le Verrier discussed for that case the possibility that the mass of Venus had been underestimated. But he reported that if Venus' mass-estimate were adjusted to account for Mercury's perihelion motion, it would have to be increased by about 10% of its existing estimate, and if true, that increase would produce much extra perturbation on the motion of the earth, something that could not be reconciled with available observations of the earth's perturbations (measured by solar observations). So Le Verrier did not see any adjustment of the mass-estimate of Venus as a viable explanation or solution for the Mercury problem.
Then the other main class of conceivable explanation for Le Verrier would put the extra unaccounted mass somewhere nearer to the Sun than Mercury's orbit. Le Verrier recognised that a simple form of this class of explanation could involve an intra-mercurial planet. But in 1859 he did not consider this explanation to be likely, because such a planet ought to be bright, and thus, even if too close to the Sun to be usually seen, it ought still to be occasionally noticeable at solar eclipses and/or by occasional passages across the sun's disk -- and nothing of the kind had been observed.
The possibility that did appeal to Le Verrier in his paper of 1859, as a possible explanation that would avoid such difficulties, was not an intra-mercurial planet, but rather a 'series of corpuscles with orbits between Mercury and the Sun' ('une série de corpuscules circulant entre Mercure et le Soleil' ref.1, p.382), forming some kind of ring (ref.1 p.383).
(Le Verrier's contemporary 'Théorie du mouvement de Mercure' (1859, Annales de l'Observatoire de Paris, v.5, at pp.105-6) examined the observations in much greater detail before expressing the same conclusion, that an intra-Mercurial planet is unlikely and that more likely was a ring of matter closer to the Sun than Mercury's orbit.)
In spite of le Verrier's doubts of 1859, the idea of an intra-Mercurial planet attracted enthusiasm, and shortly afterwards, a French amateur astronomer Lescarbeault claimed to have seen it. Le Verrier for a time thought this observation plausible. But not only were there no credible corroborations: the credibility of Lescarbeault's report itself was also very soon undermined by another French observer, Liais, who in 1860 made public that he had been closely studying the sun over the same time-period as that of Lescarbeault's claimed observation, and he had seen nothing of the kind. The idea of an intramercurial planet lost credibility in serious discussions (discussed in ref.2 (Roseveare, 1982) at p.25-6, also in ref.4.)
Closely related to Le Verrier's idea of a ring of 'corpuscles' was
-- Seeliger's (1906) hypothesis of bands of diffuse intra-Mercurial matter, of which the possible existence received some observational corroboration through the zodiacal light -- discussed in ref.2.
Other hypotheses included
-- solar oblateness (quantitatively insufficient to produce the extra perihelion motion)
-- a short-lived idea that the exponent in the inverse-square law should be adjusted upwards slightly from 2 to account for Mercury's perihelion motion: this was proposed 1894 by Asaph Hall, modifying an earlier suggestion of Simon Newcomb, but it was refuted by E W Brown (1903) (ref.3), who showed that such an adjusted gravitational law would be incompatible with the observed motion of the moon's perigee. W de Sitter later concurred with Brown's conclusion.
These and other possibilities are reviewed and discussed in ref.2
(N T Roseveare, 1982) and ref.4 (J Earman and M Janssen, 1993).
Of the possibilities discussed, @Conifold's comment looks right that the
'most prominent' of them were the Seeliger hypothesis and the alteration of the exponent in the inverse square law.
But none of these pre-relativistic possibilities provided viable explanations of the extra (unaccounted-for) precession of Mercury's
perihelion. The assessment of all of them in a review by Crommelin in 1920 (ref.5) was that "Asaph Hall's suggestion [to alter the exponent in the law of gravitation] breaks down utterly for the Moon, and every other suggestion introduces as many difficulties as it removes".
References:
1: U. Le Verrier (1859), "Lettre de M. Le Verrier à M. Faye sur la théorie de Mercure et sur le mouvement du périhélie de cette planète", Comptes rendus hebdomadaires des séances de l'Académie des sciences (Paris), vol. 49 (1859), pp. 379-383 (in French). ('Letter from M Le Verrier to M Faye about the theory of Mercury['s motion] and about the motion of its perihelion'; 'Weekly reports of meetings of the Academy of Sciences (Paris).)
2: N T Roseveare (1982) "Mercury's Perihelion from Le Verrier to Einstein." (OUP), esp. chapters 2-4.
3: E W Brown (1903) "On the verification of the Newtonian law", Monthly Notices of the Royal Astronomical Society 63, 396-7. (Brown concluded that the Hall/Newcomb proposed adjustment to the exponent in the
inverse-square law of gravitation to try and account for Mercury's excess perihelion advance was incompatible with the rate of advance of the moon's perigee, and that any adjustment compatible with the moon's motion would only generate too little extra motion to solve the Mercury problem. Brown's conclusion was agreed and adopted by W de Sitter (1913) in "Some problems of astronomy (VII The secular variation of the elements of the four inner planets)", The Observatory, 36, 296-303.)
4: J Earman and M Janssen (1993) "Einstein's Explanation of the Motion of Mercury's Perihelion", pp.129-172 in "The Attraction of Gravitation : New Studies in the History of General Relativity" (ed. by J Earman, M Janssen, J D Norton), esp. at 132-135.
5: A C D Crommelin (1920), "The Motion of Mercury's Perihelion", Journal of the British Astronomical Association vol.30, 123-127.
