Gibbs' inequality

$$-\sum\limits_{i=1}^n p_{i} \cdot \log{p_{i}} \le -\sum\limits_{i=1}^n p_{i} \cdot \log{q_{i}}$$ is such a popular thing that I cannot find where it was introduced.

My findings

I looked at "Elementary principles in statistical mechanics" (1902), and the most similar thing I've found is the theorem about the maximum entropy of uniform distributions:

Theorem IX. A uniform distribution of a given number of systems within given limits of phase gives a less average index of probability of phase than any other distribution.

meaning (index of probability is the natural logarithm of the probability) $$\int\limits_{x \in X} \left(\Delta\eta\left( x \right) + \eta \right) \cdot e^{\Delta\eta\left( x \right) + \eta} dx > \int\limits_{x \in X} \eta \cdot e^\eta dx,$$ where $$\int\limits_{x \in X} e^{\Delta\eta\left( x \right) + \eta} dx = \int\limits_{x \in X} e^\eta dx = 1.$$

In "On the equilibrium of heterogeneous substances" (1874) I see

In the first place, if the system is in a state in which its entropy is greater than in any other state of the same energy, it is evidently in equilibrium, as any change of state must involve either a decrease of entropy or an increase of energy, which are alike impossible for an isolated system.

but I can't find formulae looking like the Gibbs' inequality.


Had Gibbs actually shown the inequality or it's just called after his name as a sign of respect?

If he had, in which work can it be found and how is it formulated?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.