I am curious to understand the history of QM. In particular, how did the physicists conclude that observables can be treated as operators, or the use of complex state spaces, or that Eigenvalues of the operators are the only values that can be observed. Can somebody recommend a book that discusses this historically?
There are two very comprehensive books covering the whole story
Jagdish Mehra and Helmut Rechenberg, The historical development of quantum mechanics, in 3 volumes, and
Mitsuo Taketani, Formation and logic of quantum mechanics, in 3 volumes, World Sci. 2001 (translated from the Japanse).
Some good shorter books are
Jim Baggott, The Quantum story. A history in 40 moments, Oxford, 2011, and other books of the same author, for example The Quantum cookbook, Oxford, 2020.
There is a nice collection of English translations of the most important original papers, with commentaries:
B. L. Van der Waerden, Sources of quantum mechanics, North Holland 1967.
There are also books and papers that cover history specific parts or aspects of QM:
Sin-Itiro Tomonaga, The story of spin, U. Chicago, 1997 (translated from Japanese), unlike other listed books, this is written by one of the key developers of the subject. And the article by
Shlomo Sternberg, A history of 19th century spectroscopy, Appendix F to his book Group theory and physics, Cambridge, 1994. (This is one of the best texts on history of science that I ever read).
I think the best book for this is B. L. van der Waerden's "Sources of Quantum Mechanics". In short, the book explains several articles of early quantum mechanics in a way that a modern reader can follow, starting from an Einstein paper from 1916 up to a Dirac paper from 1926, together with a reprint of all the original papers translated to english.
This work is remarkable because by following the papers one can see how Heisenberg's introduction of matrix operators was very natural given the classical and semi-classical results previously published.
One note, the book concerns the operator approach, and as best as I can remember, says nothing of the concurrent approach that led to Schrodinger's wave theory