There is a detailed account of Riemann's physical thinking in Klein's Development of mathematics in the 19th century, see also Bernhard Riemann’s ‘Dirichlet’s Principle’. One of the most prominent examples is the use of potential theory in his dissertation (1854) and in Theory of Abelian Functions (1857) to prove theorems about algebraic functions on Riemann surfaces, particularly what Riemann called the "Dirichlet principle".
The idea is that if one imagines a surface as conducting electricity, with batteries at the poles, then the resulting flow has to satisfy the least action principle and be harmonic, which produces the (real part) of a holomorphic function with given singularities. The original application was to the Dirichlet boundary value problem for the Laplace equation, but Riemann went far beyond that. Dirichlet principle was later criticized by Weierstrass as unjustified, he gave a counterexample to the existence of a least action minimizer for a different functional, see What was Weierstrass's counterexample to the Dirichlet Principle? on MO. But for the original context the principle was eventually vindicated by Schwarz and Hilbert.
Here is Klein's description:
"Riemann then significantly generalized this principle: instead of a bounded portion of the plane, he took a portion of a Riemann surface or even
a whole closed Riemann surface, and instead of the boundary values of $u$ he
took arbitrary relations that may exist between the boundary values of $u$ and $v$. It is impossible here to pursue this principle in all its generality.
Rather, I can only cite the results it first yielded for the theory of algebraic
functions and their integrals. According to Riemann's own statement, he found
them right at the start, in the winter of 1851/52, in connection with his
The fundamental thought experiment is: to think of the Riemann surface
as uniformly conducting electricity. This can be realized very simply: One
covers the surface with tinfoil, and, for an isolated interpenetration of the
sheets, one has combs meshing into one another along the branch cuts in such a
way that the electrical resistance [Leitungswiderstand] in the teeth of the
combs is the same as in the homogeneous tinfoil cover. At two points $A_1$, $A_2$ one places the poles of a galvanic battery of appropriate strength. A flow
develops, whose potential $u$ on the surface is everywhere single-valued and continuous and satisfies the equation $\Delta u = 0$, except that at $A_1$ and $A_2$
it becomes discontinuous like $\log r_1$ and $-\log r_2$, respectively.
With this we have won another existence theorem, which can be formulated
thus: On every closed Riemann surface there exists a continuous potential
function $u$ which at two prescribed places becomes logarithmically infinite in a definitely prescribed way.
[...] With this first existence theorem, we now have a won game. One can form
"integrals of the second kind", i.e., integrals whose only infinities single
poles of the form $1/( z-a)$; then "integrals of the first kind", which are
finite everywhere. Moreover, one can construct "algebraic functions" on the
surface in various ways: by combining integrals of the second or first kinds so that all the moduli of peridocity become zero, or else by simply differentiating--
thus $d\Pi/dz$, and so on."