The first and most famous example was the book of Archimedes which is usually called The Method (more complete title is The Method of mechanical theorems), where he uses
mechanics (statics) to compute volumes of various bodies. Unfortunately, this book was lost and found again only in the beginning of 20th century.
Meanwhile what he did there was rediscovered in 17 century by people
like Stevin, Fermat, Kepler and Cavalieri.
Archimedes (a pure mathematician of the highest rank) writes very clearly that the method is not rigorous. It took
2 centuries of development of Calculus/Analysis in 17-19 centuries to make it rigorous.
Another example is Maxwell's Treatise on electricity and magnetism, where he anticipates a lot of 20th century mathematics, including such things as differential forms, cohomology theories and extremal length. Unfortunately, mathematicians of 19th century did not appreciate Maxwell. (There is a nice paper about this, by Freeman Dyson, titled "Lost opportunities").
For example, Maxwell's discussion of electric resistance (Ch. VIII, art. 306-309) of conductors contains a method of estimating this resistance.
Maxwell mentions Rayleigh as the author of the idea.
This method was rediscovered by Ahlfors and Bers in 1950s
under the name Extremal length
which became one of the main working tools in the theory of conformal mappings. They do not refer on Rayleigh or Maxwell: the earliest predecessor they refer to is Courant, who wrote in 20th century.
Examples from 20th century are abundant: whole new areas of mathematics
were developed to lay a rigorous foundation for the insights
of Maxwell, Boltzmann and Gibbs in statistical mechanics. 20th century mathematicians are more inclined to talk to physicists and read their writings.
For example see Wikipedia articles Ergodic hypothesis and Ergodic theory.
All statistical mechanics was developed by Maxwell, Boltzman and Gibbs on
the "pysical level of rigor", and mathematicians are still busy with converting their "laws" into theorems. There is still a large gap between the laws of statistical mechanics and rigorously proved results. On the other hand, considerations from statistical mechanics led to discovery of new mathematical theorems not related to physics directly. See for example,
D. Ruelle, Is our mathematics natural? or this paper.
Such examples are really numerous.
Finally let me mention Fourier and his remarkable book Analytic theory of heat, whose main points were that a) every periodic function can be expanded
into Fourier series,
and b) every reasonable function on the real line can be represented by Fourier integral. He gives all kinds of ingenious arguments in favor of these statements (including experimental evidence with heated metal rings!). It took more than a century to mathematicians to rigorously state and justify his main assertions. Some of them were proved only recently,
Ki, Haseo and Kim, Young-One,
On the number of nonreal zeros of real entire functions and the Fourier-Pólya conjecture, Duke Math. J. 104 (2000), no. 1, 45–73.