The OP speculation is basically correct. Average density was certainly defined first, the Eureka story told about Archimedes by Vitruvius involves distinguishing gold from non-gold by comparing average densities. The same goes for speeds, Eudoxus defined them as distance over time even earlier (c. 375 BC), see Riddell, Eudoxan Mathematics and the Eudoxan Spheres, p.7. Ancient astronomy was generally preoccupied with reducing any observed motion to uniform motions.
However, aside from Platonist bias for uniformity, it was a forced move due to lack of more general technical apparatus. Reducing already presupposed that observed speeds change, it is just that accounting for that directly was not an option.
Oresme made one of the first steps towards creating such apparatus in Tractatus de configuationibus qualitatum et motuum (1353) by considering uniformly accelerated ("uniformly difform") motion graphically, where one could identify instantaneous speeds with slopes. He talked more generally about "qualities" varying ("difform") in time and space, including temperature as an example. In fact, we find the idea of relating varying qualities to "imaginary" points/lines/surfaces clearly expressed in the Tractatus, see Mumford's notes:
"Every measurable thing except numbers is
imagined in the manner of continuous quantity.
Therefore, for the mensuration of such a thing, it
is necessary that points, lines and surfaces, or
their properties be imagined. For in them, as the
Philosopher has it, measure or ratio is initially
found, while in other things it is recognized by
similarity as they are being referred to by the
intellect to the geometrical entities. Although
indivisible points, or lines, are non-existent, still it is necessary to feign them mathematically for the measures of things and for the understanding of their ratios."
"...It is apparent that we ought to imagine a quality in this way in order to recognize its disposition more easily, for its uniformity and its difformity are examined more quickly, more easily and more clearly when something similar to it is described in a sensible figure. …Thus it seems quite difficult for some people to understand the nature of a quality which is uniformly difform. But what is easier to understand is that the altitude of a right triangle is uniformly difform."
These ideas found extensive continuation in the 17th century when, on the one hand, many qualities were not just seen but measured as varying, density, atmospheric pressure, optical density, etc, and on the other, calculus provided the means of modeling them. "Varying" could only mean varying from point to point, so the need to relate something to instants and points became even more pressing.
Cavalieri's indivisibles and Fermat's infinitesimals, provided just the something one could relate to every point, rather than to a discrete selection of them where measurements could, in practice, be made. Indeed, subdividing into infinitesimal parts and taking continuous limits was a common technique at the time. Johann Bernoulli's optical solution to the brachistochrone is a famous example, it splits optically non-uniform medium into infinitesimal layers, see Broer, Bernoulli’s light ray solution. The limiting answers were much more concise and tractable than their discrete analogs.
This happy meeting of the modeling need with the technical opportunity sealed the deal for instantaneous and point quantities over averaged approximations. Their non-geometric interpretation as limits of finite ratios came a bit later. It was favored by Newton early (by 1680s), see What was the notion of limit that Newton used?, and eventually supplanted indivisibles/infinitesimals due to worries about their coherence and justification.
