I always have been curious about this part of the History of Science. To claim that Earth is orbiting the Sun instead of the opposite is equivalent to change one absolute referential (Earth) to another one (the Sun); something Galilean relativity forbids. So how come Galileo is credited for both the statement and the theory?
For Galileo, the relativity principle was an observational principle. When he writes about the thought experiment on a moving ship where the observer cannot recognize that there is movement, it is clear to the reader that it is indeed a moving ship. So for Galileo, there can be an absolute reference system, where the Sun is at the center, and then there are all these empirically equivalent systems, such as two moving ships, whose movements cannot ever be observed. There is no contradiction between those.
This duality persisted until the early twentieth century. We find this for example in Newton, who believed in absolute space and empirically equivalent inertial systems; in Maxwell, who believed in the ether but whose equations are equivalent for moving systems; and in Lorentz, who assumed an absolute ether system but had it as a principle that the ether contracted in such a way that each inertial system was by principle empirically equivalent. It was only with Poincaré and Einstein that someone thought this duality was particularly problematic.