Is there any evidence showing that a unit circle approach was used by early mathematicians?
The modern convention, like much of modern notation, goes back to Euler's Introductio in Analysin Infinitorum (1748), chapter VIII, there is an English translation by Blanton. At the beginning of the chapter Euler writes:
"After having considered logarithms and exponentials, we must now turn to circular arcs with their sines and cosines. This is not only because these are further genera of transcendental quantities, but also since they arise from logarithms and exponentials when complex values are used. This will become clearer in the development to follow. We let the radius, or total sine, of a circle be equal to 1, then it is clear enough that the circumference of the circle cannot be expressed exactly as a rational number."
He then denotes half of the circumference by the familiar $\pi$, and gives the standard suite of trig identities.