Prograde burn and retrograde burn are quite basic maneuvers in orbital mechanics. But how did the scientists (or researchers) discover that (the mathematical interpretation)? Did they try any practical experiments or was it all a theoretical achievement? How did they discover orbits in the first place?
It was a theoretical invention. A special case, optimal elliptic transfer between two circular orbits, a.k.a. the Hohmann transfer, was described by Hohmann in The Attainability of Heavenly Bodies (1925), see Washington: NASA Technical Translation F-44, 1960 on Internet Archive. It is accomplished by two burns, one prograde one retrograde. Hohmann, apparently, drew inspiration from science fiction:
"The above concept, to impart to a body an acceleration opposing gravity by expelling parts of its mass, is not new in itself. It is contained unwittingly in Jule Verne's "Journey Around the Moon," where mention is made of rockets, which are taken along to decelerate the vehicle, and it is continuously used in Kurd Lasswitz's "On Two Planets." Here however under the very favorable assumption, that the velocity of expulsion is that of light, resulting in only a minor decrease of the vehicle's mass.
The more recent works of Goddard, Oberth and Valier have already been mentioned. Also the pioneer of aviation, Hermann Ganswiudt, has proclaimed the idea of a rocket vehicle in public talks around I890, at the same time also, the Russian Cielkowsky. Finally, Newton mentioned the possibility of travel in empty space on the occasion of a lecture concerning impulse reaction."
A similar maneuver between two elliptic trajectories, the bi-elliptic transfer, was published by Sternfeld in Sur les trajectoires permettant d'approcher d'un corps attractif central à partir d'une orbite keplérienne donnée (1934, On the allowed trajectories for approaching a central attractive body from a given Keplerian orbit), see Comptes rendus de l'Académie des sciences, Paris, 198 (1): 711–713 on Gallica.