Discarding the ungrounded speculations (for example, Columbus claimed that Earth has a "bulge"),
scientific theories which predicted deviation of Earth's shape from the sphere were proposed in he late 17th century. The correct theory was due to Newton. There was a competing theory predicting that equator is shorter than meridian (either by Descartes himself, or by his followers). To decide which theory is correct, several expeditions were sent by the French Academy to measure arcs of meridians. The first expedition did this in Peru (modern Equador) in 1735-45. Before they finished in Peru, the second expedition (to Lapland, Sweden, 1736)
returned with a conclusive result that Newton's theory was correct.
The results were published in the book of Maupertuis, La figure de la Terre
in 1738. It was translated into English in the same year.
This result was one of the first major tests of the theory of universal gravitation. (See How did Newton come up with his formula?).
EDIT. Of course this was only the beginning. Since then geodesists made many more precise measurements, and mathematicians also investigated the question in great depth. Mathematically, the question is about equilibrium of rotating liquid body; it was finally solved only in 20th century (by Chandrasekhar in 1970).