Regrading the early consideration of density as a means of defining the continuum :
William of Ockham (c. 1280–1349) brought a considerable degree of dialectical subtlety to his analysis of continuity; it has been the subject of much scholarly dispute. For Ockham the principal difficulty presented by the continuous is the infinite divisibility of space, and in general, that of any continuum. The treatment of continuity in the first book of his Quodlibet of 1322–7 rests on the idea that between any two points on a line there is a third —perhaps the first explicit formulation of the property of density— and on the distinction between a continuum “whose parts form a unity” from a contiguum of juxtaposed things. Ockham recognizes that it follows from the property of density that on arbitrarily small stretches of a line infinitely many points must lie, but resists the conclusion that lines, or indeed any continuum, consists of points. Concerned, rather, to determine “the sense in which the line may be said to consist or to be made up of anything.”, Ockham claims that “no part of the line is indivisible, nor is any part of a continuum indivisible.” While Ockham does not assert that a line is actually “composed” of points, he had the insight, startling in its prescience, that a punctate and yet continuous line becomes a possibility when conceived as a dense array of points, rather than as an assemblage of points in contiguous succession.
Subsequently, Bolzano, Cauchy, and Weierstrass all gave definitions of continuous functions and wrestled with the concept of the continuum.
Dedekind appears to be the first to recognise that density is not a sufficient condition for continuity:
Following Weierstrass's efforts, another attack on the problem of formulating rigorous definitions of continuity and the real numbers was mounted by Richard Dedekind (1831–1916). Dedekind focussed attention on the question: exactly what is it that distinguishes a continuous domain from a discontinuous one? He seems to have been the first to recognize that the property of density, possessed by the ordered set of rational numbers, is insufficient to guarantee continuity. In Continuity and Irrational Numbers (1872) he remarks that when the rational numbers are associated to points on a straight line, “there are infinitely many points [on the line] to which no rational number corresponds” so that the rational numbers manifest “a gappiness, incompleteness, discontinuity”, in contrast with the straight line's “absence of gaps, completeness, continuity.” Dedekind regards this principle as being essentially indemonstrable; he ascribes to it, rather, the status of an axiom “by which we attribute to the line its continuity, by which we think continuity into the line.” It is not, Dedekind stresses, necessary for space to be continuous in this sense, for “many of its properties would remain the same even if it were discontinuous.”
Dedekind's continuum is a point set.
It is to be noted that Dedekind does not identify irrational numbers with cuts; rather, each irrational number is newly “created” by a mental act, and remains quite distinct from its associated cut.
Source: Stanford Encyclopedia of Philosophy
EDIT
According to the arXiv paper On the History of the Number Line, Bolzano's algebraic approach to the problem used the notions of equality, order, and density.
He introduced the notion of a measurable number, relations described as “equal to”, “greater than”, “less than”; asserted density (pantachisch) of a set of real numbers.
The paper makes no mention of density with reference to Cauchy or earlier mathematicians - e.g., no mention of Ockham at all. Also, Bolzano's term pantachisch does not translate using google's translate program.
Regarding Leibniz's view of density (mentioned in the comments),
One might be tempted to attribute to Leibniz the modern view that density is not a sufficient condition for continuity. (...) There is, however, ample evidence that for Leibniz, dense quantities, namely that between any two numbers there exists a third, are continuous. For Leibniz,
"Continuous, is a whole such that its co-intergrating (cointegrands) parts, (i.e., such that taken together coincide with the whole) have something in common, and moreover such that if its parts are not redundant (i.e., they have no part in common, that is, there aggregate is equal to the whole) they have at least a term [terminum] in common."
Source : Leibniz and Clarke - A Study of Their Correspondence, by Ezio Vailati.