I'll focus on the geometry of Yang-Mills theories specifically, but as Conifold's answer points out, gauge theories were studied geometrically long before the work of Yang and Mills.
The foreward to volume 5 of Atiyah's collected works (on gauge theories) contains some historical comments on this from the mathematics side. You can read it here. This is probably somewhat later than what you're looking for, since it has more to do with the later study of the deep geometric properties of Yang-Mills theory than the (comparatively) simple interpretation of Yang-Mills as an action on the space of connections.
Atiyah says his own interests in the subject began in 1977, when he says interest in gauge theories among mathematicians was just beginning to start (citing Yang's influence in mathematical circles). That matches pretty well his writings. The first writing included in his collection is . This paper he cites with bringing many of the ideas in gauge theory to the mathematics community. In it, he shows that the multi-instanton problem can be reduced to constructing suitable vector bundles on 3-dimensional projective space. The construction was completed in . He wrote several more papers that year and in the following years on the geometry and topology of Yang-Mills fields. His (and other) papers in the late 70s are the earliest that I know of by mathematicians on the geometry of Yang-Mills theory.
By the early 80s, several other people began publishing on this topic. Some of the big names are Donaldson, Hitchin, and Witten. In particular, Donaldson's study of 4-manifolds via instantons in  proved to be of great interest. By that point, it had become clear that the Yang-Mills equations could be used to great effect for more than just physics. It's fair to say that the interest in them continued well through the 80s and in some cases to the present day.
The earlier developments before this were almost completely taken on by physicists. I know less of the story here because physicists seem less inclined to write detailed accounts of the chronology of events. It's clear that by 1977, it was already known by physicists that Yang-Mills could be viewed in terms of an action functional on the space of connections, though the geometric consequences had not been explored. (Of course, physicists had bigger problems to deal with before that, like understanding how to give gauge bosons mass and proving the renormalizability of quantum Yang-Mills.) The earliest source I know for this is by Popov in  in 1975. In this, he shows that the now-standard geometrical interpretation of Yang-Mills via principal bundles and connections yields the Yang-Mills equations. However, it's quite possible that some of the ideas there originated earlier, though I can't see anything in the citations to indicate such.
 M. F. Atiyah and R. S. Ward: “Instantons and algebraic geometry,” Comm. Math. Phys. 55 : 2 (1977), pp. 117–124.
 M. F. Atiyah, N. J. Hitchin, V. G. Drinfel’d, and Yu. I. Manin: “Construction of instantons,” Phys. Lett. A 65 : 3 (1978), pp. 185–187.
 S. K. Donaldson, "An application of gauge theory to four-dimensional topology", Jour. Differential Geometry 18 (1983), 279-315.
 Popov, D. A., "Theory of Yang-Mills Fields", 1975, Teor. Mat. Fiz. 24, 347.