From your sentence "my main motivation is still to try and get over my intense phobia of norms and inner products" I conclude that you need first of all a good book in Linear algebra itself, rather then history of linear algebra. In English, I recommend the textbook of P. Lax. There is a nice book of Dieudonné Algèbre linéaire et géométrie élémentaire (there is an English translation) which gives an exposition of high school geometry from the linear algebra point of view. Essentially this is the book which does all linear algebra in dimensions 2 and 3. That is elementary geometry, only exposed in a modern way.
On the history of linear algebra there is another book of Dieudonné, Abrégé d'histoire des mathématiques, vol. I which explains genesis of these notions.
But I have to repeat that genesis was quite complicated and convoluted, before the modern clarity and simplicity were reached. So in this particular case, I recommend NOT to follow the historical development if your problem is to understand the linear algebra itself.
Only AFTER you overcome your "phobia of norms and inner products" you may read some of this history with a profit.
EDIT. Another good book is MR1885576
Givental, Alexander
Linear algebra and differential equations.
Berkeley Mathematics Lecture Notes, 11. American Mathematical Society, Providence, RI; Berkeley Center for Pure and Applied Mathematics, Berkeley, CA, 2001.
It teaches you linear algebra in dimension 2. That is the linear algebra part covers the SAME material as middle school geometry course. Only in the modern language. If you had a geometry course at school, there must be nothing unfamiliar to you in linear algebra in dimension 2.