Could scientists of Newton's time have explored the limits of his laws of motion?

Question

As noted on this Wikipedia page Newton's laws of motion and of gravity were "verified by experiment and observation for over 200 years" and found to be a "good approximation for macroscopic objects under everyday conditions". Now, of course, we know from Einstein and quantum mechanics that while not incorrect, his laws "only" work for a certain range of inputs.

While some of those early experiments were probably carried out with the hope of proving Newton wrong, nobody knew that there were limits to their applicability (very small scales; very high speeds; very strong gravitational fields) until much later (and, as far as I know, no one even suspected that there might be such limits).

As noted in the answers to this HofS&M question, there were some known "anomalies", such as the motion of the moon or Mercury, but these were more to do with the problem of applying Newton's laws to a three-body problem than problems with the laws themselves.

The question is: if Newton, or scientists around his time somehow had an "inkling" that there actually were limits to his laws, and set off on a "directed search" to explore those limits, would they have had the technology to expose such limits? In other words, would they have been able to explore a small-enough scale or move large-enough masses, at fast-enough speeds, and measure things with enough precision to detect any discrepancies above experimental noise and be reasonably confident in concluding that Newton's laws don't hold under all circumstances?

Answer 1

At the time of Newton, the scientists could NOT detect any deviation of the Newton's laws from reality. As we know now, the only visible effect of this deviation in the Solar system is the anomalous precession of Mercury's perihelion. It was detected only in the middle of 19th century; it makes about 43 seconds per century, and it was not known at Newton's times. The problem was not with accuracy of observations, in principle they could detect the phenomenon. The problem was comparison with the theory: equations resulting from Newton's laws are notoriously difficult to solve.

Newton's theory of gravitation was not immediately generally accepted, and scientists indeed expressed doubts. Newton's theory was originally derived to explain Kepler laws, but Kepler's laws themselves only hold approximately, and Newton's main claim was that his gravity law explains not only Kepler's laws but also all observed DEVIATIONS from these laws.

It took about a century and half to verify these claims. (The problem is very difficult: Newton's theory only tells you how to WRITE the equations, but another matter is how to SOLVE them). Newton tried hard to explain the motion of the Moon (I mean the small non-uniformities of this motion) and only partially succeeded.

Mathematicians who struggled with the theory of the Moon indeed expressed doubts in the gravitation law, Clairaut, for example thought at some point that the gravity law is only an approximation to the "true law", but then he changed his opinion. The motion of the Moon was finally explained to the accuracy sufficient for navigation only in the second part of 18th century.

By the end of 19th century ALL motions in the Solar system were explained (or believed to be explainable to the accuracy of observations of that time) except this Mercury perihelion precession. The fact that Newton's gravity law does not explain the orbit of Mercury was discovered by Leverrier in the middle of 19th century. They started searching a new planet between Mercury and Sun whose perturbation could explain the effect. The search lasted long time with some claims of discovery which were later refuted. At some point even the name was given to this "new planet": Vulcan.

EDIT. Any scientific law is supposed to work only in certain range of phenomena. After the law is discovered, the efforts are directed at the precise determination of this range. This is what mathematicians and astronomers were occupied with for the century after Newton.

EDIT. One can also think whether scientists at the time of Newton could detect the deviation of light rays from stars as they pass near the Sun. The effect was observed by Eddington in 1918 and this was a major check of general relativity. The accuracy required was about 0.5-1''. Flamsteed star catalog has accuracy 4''. So this phenomenon also could not be observed in Newton's time. And the theory of light was not sufficiently developed to discuss this phenomenon.

Answer 2

One of the main criticisms of Newtons theory of Gravitation was philosophical; in that his theory incorporated action at a distance; this criticism was something that Newton was very much aware of, and that he accepted; he could not see how any man of a 'philosophical' temper could not accept this principle; however, he saw no way forward through this and hence he 'framed no hypotheses' about it. Here is how he put it himself in a letter to Bentley in 1692/3.

It is inconceivable that inanimate matter should. without the mediation of something else which is not material, operate upon, and affect other matter without mutual contact...that gravity should be innate, inherent and essential to matter so that one body may act upon another thro' a vacuum without the mediation of anything else, by and through which their action and force may be conveyed from one to another is to me so great an absurdity that I believe no man who has in philosophical matters a competent facility of thinking can ever fall into it. Gravity must be cause by an agent acting constantly according to certain laws.

This critique is of ancient provenance, Aristotle mentions it in one of his many books (I think it's in the De Caelo - On the Heavens) that it was valid is seen by the fact that the accepted theory of gravity now, that by Einstein - General Relativity - incorporates this principle.