I know that when solving geometric problem, Descartes used variables $x,y$ and derived equation such as $y^2=cy-\frac{cxy}{b}+ay-ac$. Conversely, in algebraic geometry, an arbitrary polynomial $F(X_1,...,X_n)$ is thought to define a geometric object as its zero set.
When was the first step in this direction taken?
The coordinate method may be traced to antiquity, specifically to the works of Apollonius of Perga (c. 262 – c. 190 BC) The following quotation from Carl B. Boyer,"Apollonius of Perga" (1991). A History of Mathematics (Second ed.). John Wiley & Sons, Inc. pp. 156–157. ISBN 0-471-54397-7 (as referenced in the Wikipedia article on Apollonius) explains his approach: "The method of Apollonius in the Conics in many respects are so similar to the modern approach that his work sometimes is judged to be an analytic geometry anticipating that of Descartes by 1800 years. The application of references lines in general, and of a diameter and a tangent at its extremity in particular, is, of course, not essentially different from the use of a coordinate frame, whether rectangular or, more generally, oblique. Distances measured along the diameter from the point of tangency are the abscissas, and segments parallel to the tangent and intercepted between the axis and the curve are the ordinates. The Apollonian relationship between these abscissas and the corresponding ordinates are nothing more nor less than rhetorical forms of the equations of the curves. However, Greek geometric algebra did not provide for negative magnitudes; moreover, the coordinate system was in every case superimposed a posteriori upon a given curve in order to study its properties. There appear to be no cases in ancient geometry in which a coordinate frame of reference was laid down a priori for purposes of graphical representation of an equation or relationship, whether symbolically or rhetorically expressed. Of Greek geometry we may say that equations are determined by curves, but not that curves are determined by equations. Coordinates, variables, and equations were subsidiary notions derived from a specific geometric situation; [...] That Apollonius, the greatest geometer of antiquity, failed to develop analytic geometry, was probably the result of a poverty of curves rather than of thought. General methods are not necessary when problems concern always one of a limited number of particular cases."
Probably Al-Khwarizmi. His book on algebra (the first one, from which algebra gets its name), contains diagrams, but only to illustrate (or geometrically butress) the "equations" (which are written in natural language, not symbols).
The Greeks never really got to algebra. Some of their stuff looks a bit like algebra, but it lacked the conceptual apparatus essential to genuine algebraic thinking, so it's historically inaccurate to call it algebra.
