Context: One of the key reasons Einstein first suspected space is curved by mass is due to the falling elevator thought experiment. This experiment implies that mass must curve the path of a light ray. From this, he inferred that space must be curved. But why come to that conclusion? (It could rather be for instance that space is flat and the light simply "falls" through flat space like any other object). Apparently the reason is that it was believed that light must follow a geodesic. Thus if space was flat, light would have to travel in a straight line. The thought experiment suggested light actually follows a curved path, and hence by implication, space must curved.
What first lead physicists to believe that light must follow a geodesic, and why?
I suspect the reason may be just a logical extension of Fermat's theorem for light, but this may be entirely wrong.