Could someone please explain how the concept of energy was originally conceived and how it evolved over time to our current understanding of it? Also how did people come about various ways of measuring, mathematically expressing and precisely calculating various forms of energy, like kinetic energy, potential energy, thermal energy, etc.?
The word originated with Aristotle, whose "energeia" and "entelecheia" can roughly be translated as enaction, that which makes matter move, and embodiment, that which makes matter take form, respectively. Over the middle ages the attention was focused on "impetus", roughly associated with mass times speed and usually taken to be the precursor of modern momentum, but natural philosophers did not mean mathematically phrases that look that way to us, so there was enough vagueness to say that the idea incubated both modern concepts. Kinetic energy under the name of vis viva (living force) first properly entered the stage in 17th century, when Huygens showed that it is conserved in elastic collisions, while impetus, finally mathematized by Descartes and called "quantity of motion", was not.
This last part was pointed out by Leibniz, the main proponent of vis viva far beyond mechanics only, who begged to differ on what was more deserving of the name "quantity of motion". Newton did not take kindly to the fact that a quantity hardly fundamental to his Principia got unwarranted attention, and a controversy that lasted for over half a century began. We have a detailed account of it here What was the vis viva controversy, including its philosophical aspects? It wasn't a total loss, many basic mechanical notions got clarified in the process, including the notion of work, and in 1740s Euler (1736) and D'alambert (1743) produced systematic monographs on mechanics that finally deflated the controversy. Potential energy joined the club in Lagrange's Mecanique Analytique (1788), see more in How did Newton establish the conservation laws in the Principia?
Around the start of 18th century systematic analysis of thermal energy began under the auspices of Stahl's caloric theory that treated heat as a fluid ("phlogiston"), with no inkling of course about a relation to mechanical energy. This did not stop Newcomen from inventing a new steam engine in 1712, and Watt from improving it in 1763-1775 (he also introduced a unit of power, "horsepower") After Lavoisier's work on oxygenation (1783) phlogiston theory was modified into caloric one, which remained in place until mid 19th century, Carnot even developed the basics of the theory of heat engines based on it (1824). But measurements of the mechanical equivalent of heat first performed by Rumford in 1798, and more conclusively by Joule in 1840s, originally very coldly received, the formulation of the general energy conservation law by Mayer in 1841, and the reformulation of Carnot's work by Clausius in 1850 led to the acceptance of heat as another form of energy. In 1851 Thomson already could write that “heat is not a substance, but a dynamical form of mechanical effect, we perceive that there must be an equivalence between mechanical work and heat, as between cause and effect”. See What are the major flaws of the “caloric” theory of heat?
By the time SI and CGS were undergoing major overhaul of units in 1860-1880s mechanical, thermal and electric energy were treated uniformly, see Why is there no named unit for momentum but there is one for energy?
There is a chapter "the history of the energy concept" of some 80 pages in Philip Mirowski's book More heat than light which is rather informative. The main point of the story appears to be that energy is something that is conserved, that is the concept really makes sense as an "invariant". The end of the chapter mentions various energetist and energetisms, largerly forgotten today.
Energy became the novel organizing principle of physical research, linking the previously disjunct and disparate studies of motion, light, heat, electricity, and magnetism. After this "discovery," the very science of physics was redefined to be the reduction of all phenomena to their energetic foundations, and hence implicitly the reduction of all phenomena to mechanics (Harman 1982a, p. 158). Only after 1850 did physics become the king of the sciences, usurping the throne from physical astronomy (Cannon 1978, p. 2). Energy was the reason.
Thomas Kuhn also researched the history and Mirowski reports that he came with some 16 names claiming credit for the discovery, the list of most promising being
J. R. Mayer, James Joule, Hermann von Helmholtz, and Ludwig Colding.
After a dozen descriptive pages it is pointed:
The assertion of the "discovery" of the conservation of energy actually conflates four distinct ideas: (1) the formation of a concept of energy; (2) an ontological claim that there was an energy "out there" and "in here" waiting to be found; (3) the mathematical statement that this energy is neither created nor destroyed; and (4) some procedure of justification for ideas (2) and (3). The incongruity of claiming a simultaneous discovery of energy conservation in the 1840s is that each individual so far identified only achieved a subset of the above conditions, and no individual succeeded in implementing them all.
A piece of info that is key for energy as a concept is Emmy Noether's brilliant work on symmetry and conservation laws. I have not come accross an explanation of the Noether Theorem that I can understand in detail, but I think this wikipedia article provides a good illustration:
if a physical process exhibits the same outcomes regardless of place or time, then its Lagrangian is symmetric under continuous translations in space and time respectively: by Noether's theorem, these symmetries account for the conservation laws of linear momentum and energy within this system, respectively.
Therefore, when a system behaves symmetrically in time, energy is conserved, something that occurs seldom (if ever) in reality. To me, this result suggests that energy is a mathematical formalism, an abstract concept with very interesting applications in the natural sciences, engineering, etc. but should not be treated as a thing that is actually conserved.
Note: Being a female, Noether was not allowed to become a formal college student and never received the same formal status that her coleagues had, even though the latter respected her work and tried to help her.
Here are some links that I have found: