How did Einstein arrive at his field equations without the Bianchi identities or variational formulation?

Question

When we introduce the Einstein equations in courses on General Relativity we use either the Bianchi indentity or the the variational principle to motivate the appearance of the Einstein tensor $$ G_{\alpha\beta}=R_{\alpha\beta}-\frac{1}{2}g_{\alpha\beta}R. $$ But, according to Pais ("Subtle is the Lord"), Einstein did not know the Bianchi identity, and he did not use the variational principle (introduced by Hilbert). But then: How did he get to the correct field equations?

Pais seems to suggest that Einstein initially wrote down the field equations in trace reversed form $$ R_{\alpha\beta}=8\pi G (T_{\alpha\beta}-\frac{1}{2}g_{\alpha\beta}T), $$ but if that is the case, how did he know that you should consider the trace reverse of the stress tensor?

Answer 1

Pais does not "suggest" that Einstein wrote the equation in this form, he reproduces on p.256 what Einstein wrote on November 25, 1915 with reference to his presentation to the Prussian Academy: $R^{\mu\nu}=-\kappa(T^{\mu\nu}-\frac{1}{2}g^{\mu\nu}T)$. This follows a detailed account on pp.250-255 of Einstein's struggles to implement, starting on November 4, his new/old guiding light:"I was led back to a more general covariance of the field equations, a requirement which I had abandoned only with a heavy heart in the course of my collaboration [on Entwurf] with my friend Grossmann three years earlier". These struggles included considering and discarding several prior versions of the equation, including the penultimate version $R^{\mu\nu}=-\kappa T^{\mu\nu}$, trying to get right the conservation laws and figure out the status of the unimodularity condition $\sqrt{g}=1$. It started off as a principled requirement, but ended up being a selector of convenient coordinate systems.

In fact, it is because Einstein did not know of Bianchi identities in 1915 that he could use the conservation laws $T^{\mu\nu}_{;\nu}=0$ as an additional constraint on the theory, helping him to narrow down the form of equations. As a result, he could only assert that the Bianchi identities $(R^{\mu\nu}-\frac{1}{2}g^{\mu\nu}R)_{;\nu}=0$ hold in the unimodular coordinate systems, where $\sqrt{g}=1$. It would be odd if conversely, Einstein arrived at his equation in terms of the $G$-tensor, which is artificially introduced for technical convenience, instead of the Ricci and stress-energy tensors that come up naturally. Einstein was working on extending relativity to gravity and implementing general covariance since 1907, went through a number of unsuccessful attempts, including Entwurf, looked at alternative theories, like Nordström's, lost and then regained confidence in the general covariance, and plowed through candidate equations even after that. Groundbreaking theories are often forged through trial and error, not through "motivated" textbook tricks, that get one to the "right" answer fast.