The mathematician Vladimir Arnold claimed that mathematics is a part of physics.
I am aware of Arnold's On Teaching Mathematics where he stated this view, but is there any piece of writing where Arnold, or someone else, elaborated on it?
First, on the question in the narrow sense the answer is in the negative, I am afraid, although there are some other places where Arnold expresses his views on mathematics: An apologia for Applied Mathematics in his 1996 survey, a short paper The antiscientifical revolution and mathematics, an interview with Liu for Mathematical Notices, a short note Why do we study mathematics for Russian undergraduate physics/mathematics magazine Kvant, etc.
From these one can surmise that he was deeply concerned with the social perception of mathematics, the trajectory of mathematical teaching, opposed "Bourbakization", etc. But none of them adds much to the thesis from the OP linked note, except, perhaps, to suggest that it is deliberately polemical and sociologically motivated. For example, in the Apologia he writes that "the difference between pure and applied mathematics is not scientific but only social", and:
"As a result, there came about a divorce of 'pure' mathematics from all sciences, a system of mathematical education, criminal against those taught, and the image of mathematics in the common mind was of a dangerous parasitic sect on the body of science and technology, consisting of priests of a dying religion like the druids."
Taking a broader view, and aside from Arnold's social causes, the attitude is shared, in part. For example in Are there well-known mathematicians who shared Arnold's view about mathematics as natural science? Newton and Kronecker are named as expressing something vaguely similar. As we shall see, Gauss can be added to the list. Quine opined in Two Dogmas that logic and mathematics are of a kind with natural science, more "entrenched" but still subject to revision based on the sum total of observations and experiments. Lakatos's Proofs and Refutations explicitly assimilates mathematics to the "hypothetico-deductive method" (the title is a play on Popper's Conjectures and Refutations). Magidin in Is Mathematics a Science? suggests that mathematics "follows the scientific method", etc.
However, taken literally, the thesis is hard to defend. Perhaps, mathematics of the early days of calculus can be seen as a science of bold speculation and cheap experiments, but typical works do not present conjectures confirmed by calculations and simulations, as one would expect were it true. As Arnold's own publications would attest, he was well aware of the singular role of rigorous proofs as mathematical confirmations, which is dissimilar to what we have in experimental sciences.
One author that particularly comes to mind as defending a nuanced version of Arnold's thesis, long before him, is C.S. Peirce, see e.g. his Philosophy of Mathematics, sec.10. He talks of mathematics (and logic) as staging "ideal experiments" on "diagrams" (formal models), from which general rules are inductively surmised, and describes the function of proofs employing them as checks against human error. In other words, they serve as reliable shorthands for ascertaining semantic consequence in formal models. The distinction with physics then is not just that the "ideal experiments" are cheap, but also that we exercise full control over the formation of their subjects:
"Now it is plainly not an essential part of this method in general that the tests were made by the observation of natural objects. For the immense progress which modern mathematics has made is also to be explained by the same intense interest in testing general propositions by particular cases — only the tests were applied by means of particular demonstrations. This is observation, still, for as the great mathematician Gauss has declared — algebra is a science of the eye, only it is observation of artificial objects and of a highly recondite character. [CP 1.34]
Such operations upon diagrams, whether external or imaginary, take the place of the experiments upon real things that one performs in chemical and physical research. Chemists have ere now, I need not say, described experimentation as the putting of questions to Nature. Just so, experiments upon diagrams are questions put to the Nature of the relations concerned... [CP 4.530]
Not only is it true that by experimentation upon some diagram an experimental proof can be obtained of every necessary conclusion from any given Copulate of Premisses, but, what is more, no “necessary” conclusion is anymore apodictic than inductive reasoning becomes from the moment when experimentation can be multiplied ad libitum at no more cost than a summons before the imagination... It is true that what must be is not to be learned by simple inspection of anything. But when we talk of deductive reasoning being necessary, we do not mean, of course, that it is infallible. But precisely what we do mean is that the conclusion follows from the form of the relations set forth in the premiss. [CP 4.531]
"It would be a great mistake to suppose that ideal experimentation can be performed without danger of error; but by the exercise of care and industry this danger may be reduced indefinitely. In sensible experimentation, no care can always avoid error... Thus, the necessary reasoning of mathematics is performed by means of observation and experiment, and its necessary character is due simply to the circumstance that the subject of this observation and experiment is a diagram of our own creation, the conditions of whose being we know all about." [CP 3.528, 560]