We know that a body is red because it absorbs (visible) light of all wavelengths except that for the red. But before Newton showed that white light was actually made up of seven colors how did people explain a body being red in color?
On a common sense view color requires no "explanation": the world is full of objects, they have qualities (properties) — size, shape, weight, texture, etc. — color is one of them. This was systematized in Aristotle's qualitative physics, qualities were the constituents of "substances" forming objects, and our perceptions arise from their mixtures. Ancient theories of vision mostly focused on explaining how those qualities reach our eyes. By the way, Newton's optics was corpuscular, so he did not associate colors with wavelengths. The wave optics was proposed around 1678 by Huygens, see Did wave optics anticipate quantum mechanics?, but did not become dominant until Young's double slit experiments and Fresnel's theory of diffraction in 19th century. Newton in Opticks (1704) did introduce the term "color spectrum", and concluded from experiments that color is a property of light reflected from objects, rather than of the objects themselves.
That white light splits into colored light is an interesting fact about light, but it has little to do with explaining color perception. The trichromatic (red-green-blue) theory of color vision was only proposed by Young in 1802, and postulated the existence of three types of retinal cone cells sensitive to RGB colors, which Helmholtz related to light wavelengths only in 1850. The existence of three cone types was experimentally confirmed by Svaetichin in 1956 (although the relation of their sensitivity ranges to green, blue, and especially red colors is rather loose).
It is only in the context of singling out certain qualities as more "fundamental" and reducing the rest to them somehow that one needs to "explain" how color is so reduced. This point of view became prominent only with the rise of modern science in 17th century, when the distinction between "primary and secondary qualities" was made in the context of mechanistic natural philosophy. The term itself only appeared in Locke's Essay Concerning Human Understanding (1689), but the distinction is usually traced back to Galileo, and appears in the works of other early mechanists, like Descartes and Hobbes. The "primary" qualities were, naturally, the mechanical ones. According to the Galileo Project:
"In his Assayer of 1623, Galileo explained his notion of the difference between those qualities, mostly found by touch, that are inherent in bodies (weight, roughness, smoothness, etc.) and those that are in the mind of the observer (taste, color, etc.)--in other words, the difference between what we call primary and secondary qualities. In this discussion he referred to bodies that "continually dissolve into minute particles" and stated his opinion that "for exciting in us tastes, odors, and sounds there are required in external bodies anything but sizes, shapes, numbers, and slow or fast movements." An anonymous cleric filed a report with the Inquisition in which he claimed the first citation to show that Galileo was an atomist and the second to be in conflict with the Council of Trent's pronunciations on the Eucharist. The report did not lead to any action against Galileo."
The picture of the world as "particles in motion", and the mechanistic mode of explanation of course date further back to ancient atomists, the one natural philosophy in antiquity where an "explanation" of color was called for. Their list of primary qualities was even shorter, according to Leucippus and Democritus, only shapes and sizes of atoms were "primary", while smells, tastes, and colors were our perceptions of those. For example, "sweet atoms" were round, and "green atoms" were thin and elongated, so they could pass through glass, etc. Epicurus added weight to the "primary" list, and disagreed on sensory qualities like color. According to him, they were properties not of atoms themselves but of their composites, which means that we perceive patterns in arrangement and orientation of atoms as colors, rather than the shapes of atoms themselves. The Epicurean physics is poetically described in Lucretius's De Rerum Natura (On the Nature of Things, c. 60 BC), but like all of ancient atomism it was mostly natural philosophic speculation with only loose connection to observations.
Aristotle' s explanation of color was as follows: "Firstly, white and black may be juxtaposed in such a way that by the minuteness of of the division of its parts each is invisible while their product is visible, and thus colour may be produced. This product can appear neither white nor black, but, since it must have some colour and can have neithe rof the above two,, it must be a sort of compound and a fresh kind of tint. In this way, then, we may conceive that numbers of colours over and above black and white may be produced, and that their multiplicity is due to differences in the proportion of their composition. (...)" (20-30; Aristotle De Sensu and De Memoria, text and translation with introduction and commentary by G. R. T. Ross, D. Phil. (Edin.), Cambridge: at the University Press, 1906) https://books.google.com/books?id=aZGwq6KPmfAC&pg=PA57&dq=aristotle+%2B+%22de+sensu%22+%2B+white+and+black+may+be+juxtaposed%22&lr=&as_brr=1&cd=1#v=onepage&q&f=false
The Aristotelian theory of color as being produced by mixture of darkness and light found a notable adherent in Johann Wolfgang Goethe, who in 1810 published his ``Theory of Colours", https://en.wikipedia.org/wiki/Theory_of_Colours Since this work appeared many years after Newton and Huyghens and enjoyed some popularity (the English translation appeared in 1840), this may mean that the optical theories have not been widely accepted for a long time. In particular, Goethe proposed an alternative explanation for Newton's observations of multiple colors appearing from white light through the prism. According to the above Wikipedia page: "For Newton, the prism is immaterial to the existence of colour, as all the colours already exist in white light, and the prism merely fans them out according to their refrangibility. Goethe sought to show that, as a turbid medium, the prism was an integral factor in the arising of colour." While the underlying "physical" theory is rather naive, the work of Goethe contains a lot of valid information about color perception and had influence on a number of artists.