Gregory Chaitin openly holds this position (though he might dispute the "experiment is cheap" part). He believes that the incompleteness results of Gödel, Turing and himself show that the mathematical knowledge has limits, and (according to him) that the limits are quite near. From that observation, he constructs an interesting argument (interesting in more meanings than one).
A Newtonian physicist, Chaitin says, would, after experimenting with the electro-magnetic field, be led to phenomena that cannot be explained through Newtonian mechanisms and eventually to Maxwell's equations, that is to say to new fundamental principles of physics that cannot be reduced to Newtonian physics. Likewise, the experimental investigation of the microscopic world would led a physicist who has mastered Newton's and and Maxwell's contributions to quantum mechanics and ultimately Schrödinger's equation; another principle that cannot be reduced to former principles.
Chaitin believes that mathematicians should do the same thing. When faced with a problem they cannot solve or with a new phenomenon, they should adopt new axioms, see where these new axioms lead them experimentally and advance in this way, backpedaling if necessary if one axiom turns out to lead to a contradiction. He is perfectly clear that these new axioms should absolutely not be axioms in the Euclidean sense, that is to say evident truths or foundational principles. They may be, indeed should be in his mind, complex, non-obvious propositions, like Schrödinger's equation. As an explicit example of a new axiom he recommend mathematicians adopt and whose consequences he recommend they experimentally explore, he mentions the Riemann hypothesis.
Chaitin doesn't deny that proofs are important. "It's nice if you can prove [a conjecture]" he writes "especially if the proof is short. And if you have different proofs from different viewpoints, that's very good. But sometimes you can't find a proof and you can't wait for someone else to find a proof". In that case, "[mathematics] should be pursued more in the spirit of experimental science, and [mathematicians] should be willing to adopt new principles".
Reference: Randomness in Arithmetic and The Decline and Fall of Reductionism in Pure Mathematics. Gregory Chaitin (1993).