Statistical Entropy: Boltzmann and Gibbs

The equal-probability postulate made the equilibrium of an isolated system the macrostate of largest multiplicity. Turning that counting principle into thermodynamics requires a single quantity whose maximization is equivalent to maximizing the multiplicity and which is additive across independent systems, so that it matches the extensive entropy of the macroscopic theory. The logarithm of the multiplicity is that quantity. This lesson constructs Boltzmann's entropy for an isolated system, generalizes it to Gibbs's entropy for an arbitrary ensemble, shows the two coincide when the distribution is uniform, and reads both as a measure of missing information.¹

The Boltzmann entropy

Let $Ω$ be the number of microstates accessible to an isolated system at fixed energy, volume, and particle number — the count of states in the energy shell of the previous lessons. Two demands single out its functional form as the entropy. The entropy must be a monotonic function of $Ω$ , since more accessible states should mean more entropy, and it must be additive for independent systems, to match the extensivity of thermodynamic entropy.

Consider two independent systems with multiplicities $Ω_{1}$ and $Ω_{2}$ . Because any microstate of the first can be combined with any microstate of the second, the joint system has multiplicity

Ω_{12} = Ω_{1} Ω_{2} .

Additivity demands a function $S (Ω)$ with $S (Ω_{1} Ω_{2}) = S (Ω_{1}) + S (Ω_{2})$ . The only continuous solution is the logarithm, and the constant of proportionality is fixed by matching to the thermodynamic temperature scale.

The multiplicative-to-additive conversion is the whole content of the logarithm, and it is worth seeing drawn: two boxes whose state-counts multiply have entropies that add.

The multiplicities (the state-counts, written Omega in the text and W here) of independent systems multiply while their entropies add; the logarithm converts the product of counts into a sum of entropies.

The counting is done with Stirling's approximation, which is what makes $ln Ω$ extensive. For the combinatorial multiplicities of many-body systems $Ω$ involves factorials of numbers of order $N$ , and

ln N! = N ln N - N + \frac{1}{2} ln (2 π N) + \dots \approx N ln N - N

for large $N$ , with the correction terms subextensive and negligible. Every entropy computed in the following modules — the two-state paramagnet, the Einstein solid, the ideal gas — runs through this approximation.

Multiplicity of a two-state system

The paramagnet of a later module is the cleanest illustration and supplies a second figure. Take $N$ independent spins, each up or down, with $n$ up. The multiplicity is the binomial coefficient

Ω (n) = (n N) = \frac{N !}{n ! ( N - n )!},

the number of ways to choose which spins point up. Its logarithm, via Stirling with $f = n / N$ , is

ln Ω (n) = - N [f ln f + (1 - f) ln (1 - f)],

the binary entropy function times $N$ . It is maximal at $f = \frac{1}{2}$ (equal numbers up and down), where $Ω$ takes its largest value $(N /2 N)$ , and it falls off as a Gaussian of width $Δ n = \frac{1}{2} N$ about the peak — the sharpness established in the last lesson, here for a concrete count.

The multiplicity of N spins with n pointing up peaks sharply at the equal split n = N/2, with a Gaussian width proportional to the square root of N; the vast majority of arrangements sit near equal populations.

The Gibbs entropy

Boltzmann's formula presumes a uniform distribution over the accessible states — the microcanonical situation. A system in contact with a reservoir is not uniform: its microstates carry unequal probabilities $p_{i}$ . Gibbs's entropy extends the definition to any probability distribution over microstates.

The Gibbs entropy reduces to the Boltzmann entropy exactly when the distribution is uniform over $Ω$ accessible states. Setting $p_{i} = 1/Ω$ for each of the $Ω$ accessible microstates and zero otherwise,

S = - k i = 1 \sum Ω \frac{1}{Ω} ln \frac{1}{Ω} = - k Ω \cdot \frac{1}{Ω} (- ln Ω) = k ln Ω.

The two entropies agree on the microcanonical ensemble, and the Gibbs form is the general one, valid for the canonical and grand-canonical ensembles where the probabilities are not uniform. It is additive in the same sense: for two independent systems the joint distribution factorizes, $p_{ij} = p_{i}^{(1)} p_{j}^{(2)}$ , and

S_{12} = - k ij \sum p_{i}^{(1)} p_{j}^{(2)} (ln p_{i}^{(1)} + ln p_{j}^{(2)}) = S_{1} + S_{2} .

Entropy as missing information

The Gibbs form is identical, up to the constant $k$ , to the quantity Shannon introduced as the measure of the information missing from a probability distribution — the average number of yes/no questions needed to determine the microstate. A distribution concentrated on one microstate ( $p_{1} = 1$ , the rest zero) has $S = 0$ : the state is known, nothing is missing. A distribution spread evenly over $Ω$ states has the maximum entropy $k ln Ω$ available to that support: the state is maximally uncertain. Between these extremes the entropy measures how spread the distribution is.

The Gibbs entropy measures the spread of a distribution. A broad distribution over many microstates has large entropy; as probability concentrates onto a few states the entropy falls, reaching zero for a distribution supported on a single state.

This reading resolves the apparent subjectivity of the equal-probability postulate. The uniform distribution is the one that assumes the least beyond the constraints — it maximizes the missing information consistent with what is known. Assigning any other distribution would encode information about the microstate that the macroscopic constraints do not supply.

The second law as the drift to maximum multiplicity

Bring two isolated systems into contact so they exchange energy at fixed total $E = E_{1} + E_{2}$ . Before contact each was separately in equilibrium; the combined constraint now allows the energy to redistribute. By the equal-probability postulate the joint system explores all microstates compatible with the total energy, and the probability of a split $(E_{1}, E - E_{1})$ is proportional to the joint multiplicity

Ω_{12} (E_{1}) = Ω_{1} (E_{1}) Ω_{2} (E - E_{1}) .

Taking the logarithm, the equilibrium split maximizes

ln Ω_{12} = ln Ω_{1} (E_{1}) + ln Ω_{2} (E - E_{1}),

that is, it maximizes the total entropy $S_{1} + S_{2}$ . Setting the derivative with respect to $E_{1}$ to zero,

\frac{\partial ln Ω _{1}}{\partial E _{1}} = \frac{\partial ln Ω _{2}}{\partial E _{2}} ⟺ \frac{\partial S _{1}}{\partial E _{1}} = \frac{\partial S _{2}}{\partial E _{2}},

which is the equality $1/ T_{1} = 1/ T_{2}$ once temperature is defined by $1/ T = \partial S / \partial E$ . The systems reach a common temperature, energy flows so as to increase the total number of accessible microstates, and the second law is the statement that this number does not decrease. The next module derives temperature, pressure, and the chemical potential in full from this maximization; the point here is that the second law is a counting statement about multiplicity.

Maximizing the Gibbs entropy under constraints

The same maximization principle, applied to the Gibbs entropy with constraints imposed by Lagrange multipliers, generates the equilibrium distributions of the following modules. Seek the distribution ${p_{i}}$ that maximizes $S = - k \sum_{i} p_{i} ln p_{i}$ subject to normalization and a fixed mean energy,

i \sum p_{i} = 1, i \sum p_{i} E_{i} = ⟨ E ⟩ .

Introduce multipliers $α$ for normalization and $β$ for the energy, and extremize

L = - i \sum p_{i} ln p_{i} - α (i \sum p_{i} - 1) - β (i \sum p_{i} E_{i} - ⟨ E ⟩),

where $k$ has been absorbed into the multipliers. Differentiating with respect to $p_{i}$ ,

\frac{\partial L}{\partial p _{i}} = - ln p_{i} - 1 - α - β E_{i} = 0 ⟹ p_{i} = e^{- 1 - α} e^{- β E_{i}} .

Normalization fixes the prefactor and gives the canonical distribution

p_{i} = \frac{e ^{- β E_{i}}}{Z}, Z = i \sum e^{- β E_{i}},

with $Z$ the partition function. The multiplier $β$ is identified with $1/ k T$ by matching the resulting entropy to the thermodynamic $\partial S / \partial E$ , recovering the Boltzmann factor of the first lesson from a maximum-entropy principle rather than a reservoir argument. The two derivations meet in the canonical module.

Maximum-entropy inference. With no constraint beyond normalization the flat distribution maximizes the Gibbs entropy; adding a mean-energy constraint tilts it into the exponential Boltzmann form.

The entropies in one view

The three notions of entropy that recur through the course are one quantity seen under different constraints.

Entropy	Form	Applies to	Reduces to
Boltzmann	$S = k ln Ω$	isolated system, uniform $p_{i}$	itself (microcanonical)
Gibbs	$S = - k \sum_{i} p_{i} ln p_{i}$	any ensemble	$k ln Ω$ when uniform
Shannon	$H = - \sum_{i} p_{i} log_{2} p_{i}$	any distribution	$S / (k ln 2)$

The Boltzmann entropy counts states for the isolated system, the Gibbs entropy generalizes to systems in contact, and the Shannon measure supplies the information-theoretic reading that justifies the equal-probability postulate as least-biased inference. All three grow toward the same maximum as a distribution spreads, and the second law is their common increase. The following module puts the Boltzmann form to work, deriving the microcanonical thermodynamics of the ideal gas and the two-state paramagnet.

Reif, Fundamentals of Statistical and Thermal Physics, §3.3–3.5 — the definition of entropy as $k ln Ω$ , its additivity, and the approach to equilibrium as the increase of the total number of accessible states. ↩
Schroeder, An Introduction to Thermal Physics, §2.6 and §3.1 — the multiplicity of combinatorial systems, Stirling's approximation, and the definition $S = k ln Ω$ with $1/ T = \partial S / \partial E$ . ↩
Kardar, Statistical Physics of Particles, §4.3, and Pathria & Beale, Statistical Mechanics, §3.3 — the Gibbs entropy $- k \sum p_{i} ln p_{i}$ , its reduction to $k ln Ω$ for a uniform distribution, and the maximum-entropy derivation of the canonical distribution by Lagrange multipliers. ↩