I'll try with some calculations : please, check it and the formulae used ...
A solid ball with a mass $m$ of $1$ kg falls (with the usual approxiamtions : no drag, etc.) with an acceleration $a$ that is about $10 \ m/sec^2$.
This means that falling from a tower $80$ meters heigh, it will touch ground after $4$ sec, with a final velocity of about $40 \ m/sec$.
You are right : in the same time, the Earth will "fall towards" the ball, attracted by the same gravitational force.
The mass $M$ of the Earth is about $6 \times 10^{24} \ kg$.
The acceleration $A$ of the Earth that is proportional to $a$ as the reciprocal of the masses; i.e. :
$A = a \times m/M = 10/(6 \times 10 ^{24}) \approx 2 \times 10 ^{-24} \ m/sec^2$.
After $4$ seconds, the Earth reachs a velocity of fall of $8 \times 10 ^{-24} \ m/sec$ and it has traversed a space $s \approx 16 \times 10 ^{-24} \ m$.
This means that, due to the reciprocal attraction, the two bodies will touch each other in slightly less than $4 \ sec$ and that the "real" space traversed by the falling ball with respect to the Earth is about :
$80$ meters minus the space traversed by the "falling" Earth during the short time of the fall.
It can be useful to recall that :
the size of atoms is measured in picometers : trillionths ($10 ^{-12}$) of a meter.
If the ball has a mass $m'$ of $1000 \ kg$, the force with which it "pulls" the Earth will be $1000$ times greater, producing an acceleration $A' \approx 2 \times 10 ^{-21} \ m/sec^2$.
This implies that in this second case the space traversed by the Earth duing the fall of the heavier ball will be $s' \approx 16 \times 10 ^{-21} \ m$.
According to Aristotle (Physiscs, Book VII) the "law" of dynamics is :
"if a power $\alpha$ moves a body $\beta$ during time $\delta$ for a distance $\gamma$, then an equal power $\alpha$ will move a body half of $\beta$ along a distance twice as $\gamma$ in the same time".
In an anachronistic way, we can say :
$F \propto V$.
Thus, if we apply this law to "our model" of free fall, with the weight of the body as the force, we have that - assuming that after an initial short time of acceleration the falling body will reach a constant "terminal velocity" - in the second case the acquired speed $v'$must be $1000$ times the first one : $v$.
This means that after $4$ seconds the heavier ball has traversed a space : $s'= v' \times t$, i.e. $s' = 1000 \times v \times t = 1000 \times s$, where $s$ is the space traversed by the lighter body after a fall of $4$ seconds.
Conclusion
As you can see the "same factor" : $1000$ acts in a completely different way in the two models.
Due to the huge mass of the Earth, for bodies of "normal" size (meaning body of our daily life experience) it has no experimentally verifiable effect on the behaviour of different bodies falling due to the gravitational force.
In the Aristotelian model, that factor has an evident experimentally verifiable effect on the behaviour of different bodies falling due to the "tendency towards the centre".
This is exactly where is the "conceptual" difference : to perform some sort of experimental test.
Thus, the answer to :
Was Aristotle really wrong about gravity?
is a definitive YES if we try to answer the question from the point of view of modern science, a point of view that is not that of aristotelian natural philosophy.
If instead we want to compare two different qualitative "worldviews", things are different (see at least the philosophical debate involving : Thomas Kuhn, The Incommensurability of Scientific Theories, Scientific Revolutions, Historicist Theories of Scientific Rationality, Imre Lakatos and Paul Feyerabend).
Notes
i) In our computations we made approximations; approximation is a modern concept.
Without precise mathematical laws there are no possible approximations.
Aristotle's "natural laws" are not approximations in the modern sense; they are qualitative description of facts, like the one made by the same Aristotle on botany and zoology (whih were impressively accurate, by the way).
ii) Writing the "aristotelian equation" we make an "historical mistake" : he never thinked in terms of mathematical formuale.
Please, note that the mathematics used by Galileo in his analysis of the free falling body problem was only the theory of proportions of Eudoxus/Euclid, that was already available in Aristotle's time.
Thus, the "tools" available to ancient natural philosophers were more or less the same compared to the ones available to Galileo (not so with Newton ...).
Postscriptum
The above "experiment" was alredy discussed by Galileo, in advance of the correct formulation of the law of universal gravitation. See :
SALV. We infer therefore that large and small bodies move with the same speed provided they are of the same specific gravity.
SIMP. Your discussion is really admirable; yet I do not find it easy to believe that a bird-shot falls as swiftly as a cannon ball.
SALV. Why not say a grain of sand as rapidly as a grindstone? But, Simplicio, I trust you will not follow the example of many others who divert the discussion from its main intent and fasten upon some statement of mine which lacks a hair’s-breadth of the
truth and, under this hair, hide the fault of another which is as big as a ship’s cable. Aristotle says that “an iron ball of one hundred pounds falling from a height of one hundred cubits reaches the ground before a one-pound ball has fallen a single cubit.” I say that they arrive at the same time. You find, on making the experiment, that the larger outstrips the smaller by two finger-breadths, that is, when the larger has reached the ground, the other is short of it by two finger-breadths; now you would not hide behind these two fingers the ninety-nine cubits of Aristotle, nor would you mention my small error and at the same time pass over in silence his very large one. Aristotle declares that bodies of different weights, in the same medium, travel (in so far as their motion depends upon gravity) with speeds which are proportional to their weights; this he illustrates by use of bodies in which it is possible to perceive the pure and unadulterated effect of gravity, eliminating other considerations, for example, figure as being of small importance, influences which are greatly dependent upon the medium
which modifies the single effect of gravity alone.Thus we observe that gold, the densest of all substances, when beaten out into a very thin leaf, goes floating through the air; the same thing happens with stone when ground into a very fine powder.
But if you wish to maintain the general proposition you will have to show that the same ratio of speeds is preserved in the case of all heavy bodies, and that a stone of twenty pounds moves ten times as rapidly as one of two; but I claim that this is false and that, if they fall from a height of fifty or a hundred cubits, they will reach the earth at the same moment.