Floating Point

Integers are exact but cramped: a 32-bit int cannot reach a billionth or a trillion. Floating point gains range by devoting bits to an exponent, storing a number as a sign, a fractional significand, and a scale — binary scientific notation. In exchange, most values are stored approximately, and the rules governing that approximation are subtle. This lesson lays out the IEEE-754 formats, decodes and encodes a concrete value, and explains the rounding behavior that makes 0.1 inexact.

The IEEE-754 form

Every IEEE-754 number is built from three fields — a sign bit $s$, an exponent field $exp$ of $e$ bits, and a fraction field $frac$ of $f$ bits — packed sign-first, exponent next, fraction last. The two standard widths fix $e$ and $f$:

The general value, when the number is normalized, is

Two design choices make this compact. The significand $M=1.f$ has an implied leading 1 that is never stored, gaining a free bit of precision. And the exponent is stored biased — as $E+\text{Bias}$, an unsigned field — so that the bit patterns sort in the same order as the numbers, letting integer comparison hardware compare floats.

Three regimes: normalized, denormalized, special

The exponent field does double duty as a selector among three encodings. The all-zeros and all-ones patterns are reserved; everything between is normalized.

• Normalized ($exp$ neither all-0 nor all-1): the usual case above, $M=1.f$ with implied leading one, $E=exp-\text{Bias}$.
• Denormalized ($exp=0$): the leading digit becomes $0$, so $M=0.f$, and the exponent is fixed at $E=1-\text{Bias}$. This represents zero (when $frac=0$, with a signed $\pm 0$) and the tiny values that fill the gap between zero and the smallest normalized number, giving gradual underflow.
• Special ($exp$ all-1): $frac=0$ encodes $\pm \infty$ (from overflow or division by zero); $frac\ne 0$ encodes NaN, the result of an undefined operation like $0/0$ or $\sqrt{-1}$.

The denormalized region's purpose is continuity: without it there would be a sudden jump from the smallest normalized number straight to zero, and many small differences would flush to zero. Denormals fill that gap with evenly spaced values so $x-y=0$ if and only if $x=y$. The numbers make the design visible in single precision. The smallest normalized value is $1.0\times 2^{-126}\approx 1.18\times 10^{-38}$; the denormals below it march down in uniform steps of $2^{-149}$, from $(1-2^{-23})\times 2^{-126}$ just under the boundary to the smallest positive value of all, $2^{-149}\approx 1.4\times 10^{-45}$. As a value decreases through this band it loses fraction bits to leading zeros, trading precision for range — gradual underflow, and the reason the newer IEEE vocabulary calls these values subnormal.

Density: precision is relative, not absolute

Fixed-point spacing is what integers have: neighbors sit exactly $1$ apart everywhere. Floating point instead keeps relative spacing constant. Within one binade $[2^E,2^{E+1})$ the $2^f$ fraction patterns are evenly spaced $2^{E-f}$ apart, and stepping into the next binade doubles the gap. A toy format with three fraction bits shows the shape:

The same geometry at full width explains a fact worth remembering: single precision represents every integer up to $2^{24}$ exactly, then starts skipping. At $2^{24}$ the gap between neighbors reaches $2$, so $16,777,217$ has no float. A double, with $52$ fraction bits, holds every integer through $2^{53}$ and then develops the same holes. Precision is about one part in $2^{f+1}$ of the magnitude, wherever on the line the value lives.

A worked decode and encode

Take the single-precision value $V=6.5$. To encode it, first write it in binary: $6.5=110.1_2=1.101_2\times 2^2$. Read off the three fields. The sign is $0$ (positive). The exponent is $E=2$, so the stored field is $exp=E+127=129=10000001_2$. The fraction is the bits after the implied leading one: $frac=1010000{\dots }_2$ (23 bits, trailing zeros).

To decode in reverse, read $s=0$, $exp=129$ so $E=129-127=2$, and $M=1.101_2=1+\frac{1}{2}+\frac{1}{8}=1.625$. Then $V=1.625\times 2^2=6.5$, recovering the original. The packed 32-bit word is 0x40D00000.

A tiny 8-bit toy format makes the three regimes visible in numbers small enough to check by hand. Give it one sign bit, a 4-bit exponent (bias $2^{4-1}-1=7$), and a 3-bit fraction. Three encodings from this format, one per regime, show the selector at work.

The denormal row is the instructive one: because $exp=0$ pins the exponent at $E=1-\text{Bias}$ (not $0-\text{Bias}$) and drops the implied leading one, the values slide continuously down toward zero instead of leaving a gap. Its neighbor 0 0000 001 decodes to $0.001_2\times 2^{-6}=2^{-3}\times 2^{-6}=2^{-9}$, the smallest positive value this toy format can represent, and 0 0000 000 is $+0$. Line them up and the denormals fill the interval below the smallest normal $2^{-6}=0.015625$ with evenly spaced steps of $2^{-9}$.

Rounding: four modes, one default

Most reals cannot be hit exactly, so every operation ends by rounding its true result to a representable neighbor. IEEE-754 defines four ways to choose the neighbor:

• Round-to-nearest-even (the default): pick the closer neighbor; on an exact tie, pick the one whose last fraction bit is $0$.
• Round-toward-zero: truncate, dropping the excess bits. This is the mode C uses for float-to-int casts.
• Round-down (toward $-\infty$) and round-up (toward $+\infty$): the directed pair, useful for bracketing a true result between guaranteed bounds.

Rounding the decimal values $1.4$, $1.6$, $1.5$, $2.5$, and $-1.5$ to integers separates the four:

At the bit level, round-to-even looks at the tail, the bits below the last position kept, and compares it against one half of the last kept unit, the pattern $100\dots$ Rounding to two fraction bits (nearest quarter):

Why bother with the even rule? Always rounding ties the same direction (say, up) introduces a systematic drift across a long sum; ties to even go up half the time and down half the time, and the bias cancels. The same principle in decimal is the familiar round half to even (banker's rounding): $0.5$ rounds to $0$, but $1.5$ and $2.5$ both round to $2$.¹

Rounding twice is not rounding once

A subtle corollary: rounding to an intermediate precision and then to the final one can give a different answer than rounding directly, even when every step is correctly rounded. Take $1.0101_2=1.3125$ and round it to one fraction bit. Directly, the tail $101>100$ is above half, so the result is $1.1_2=1.5$. Now go through two fraction bits first: the tail $01$ rounds $1.0101_2$ down to $1.01_2=1.25$, and $1.01_2$ is an exact tie at one fraction bit, so the even rule picks $1.0_2=1.0$. Direct rounding gives $1.5$; double rounding gives $1.0$. The first rounding discarded the information that the value sat above the halfway point.

This occurs in real hardware. The x87 floating-point unit that predates x86-64's vector registers computes everything in 80-bit extended precision and rounds again when a result spills to a 64-bit double, so the same C expression could yield different bits depending on when the compiler spilled. The same effect occurs in float f = d1 + d2;, which rounds the sum to double and then rounds that to float.²

Why 0.1 is not exact

A binary fraction can represent exactly only those values whose denominator is a power of two. The decimal $0.1=1/10$ has a factor of $5$ in its denominator, so in binary it is a repeating expansion, just as $1/3=0.333\dots$ repeats in decimal:

Stored in a 23- or 52-bit fraction, this infinite tail is truncated and rounded, so the value held is near $0.1$ but not equal to it. The stored double is actually $0.1000000000000000055511\dots$, which is why the classic test fails.

#include <stdio.h>
int main(void) {
  double sum = 0.1 + 0.2;
  printf("%.17f\n", sum);        /* 0.30000000000000004 */
  printf("%d\n", sum == 0.3);    /* prints 0: NOT equal  */
  return 0;
}

Neither $0.1$, $0.2$, nor $0.3$ is exactly representable, and the rounding errors in $0.1$ and $0.2$ do not cancel against the rounding error in the stored $0.3$. This is not a bug in any one machine; it is inherent to base-2 fractions, and the practical rule is to never compare floats with ==, testing instead whether the difference is below a small tolerance.³

Addition that refuses to associate

Because every operation rounds, algebraic identities that hold for reals fail for floats. The most consequential loss is associativity. Floating-point addition is still commutative, and it is monotonic, but the grouping of a sum changes its value:

float a = (3.14f + 1e10f) - 1e10f;   /* 0.0f     */
float b = 3.14f + (1e10f - 1e10f);   /* 3.14f    */

The mechanism is absorption, and it is visible at the bit level. To add two floats the hardware must first align their radix points, shifting the smaller operand's significand right by the difference in exponents. Here $10^{10}\approx 2^{33.2}$ and $3.14\approx 2^{1.65}$, an exponent gap of about $32$; shifting a 24-bit significand right by $32$ slides every bit past the edge of the fraction, so the addition contributes nothing and the parenthesized sum is exactly $10^{10}$. Subtracting $10^{10}$ then leaves $0$. Group the other way and $10^{10}-10^{10}=0$ first, preserving $3.14$ intact.

The practical consequence: the order in which a program sums a list changes the answer, so compilers do not reorder floating-point arithmetic unless explicitly told correctness does not matter, and numerical code that adds values of wildly different magnitudes must choose the order that loses less.⁴

Crossing between int and float

C converts freely between the integer and floating families, and each direction has a failure mode worth knowing cold.

• int to float rounds. A float fraction holds $23$ bits, so integers above $2^{24}$ are not all representable: 16777217, which is $2^{24}+1$, rounds to the even neighbor $16777216.0$. Conversion to double is exact for every int ($52>31$) but rounds for large long values.
• float to int truncates, rounding toward zero: 3.9 becomes 3, -3.9 becomes -3.
• Out-of-range conversions have no answer. Casting 1e10f or a NaN to int is undefined; x86-64 hardware returns the integer indefinite pattern 0x80000000, which is $TMin$, regardless of sign.

int   big  = 16777217;      /* 2^24 + 1 */
float f    = big;           /* rounds to 16777216.0f */
int   back = (int) f;       /* 16777216: the round trip lost the 1 */
int   t    = (int) -3.9f;   /* -3: truncation toward zero */
int   boom = (int) 1e10f;   /* undefined; x86-64 yields INT_MIN */

The round trip is the case that surfaces in real systems: an integer stored in a floating type comes back changed once it crosses the fraction width, $2^{24}$ for float and $2^{53}$ for double — the latter being why integer identifiers silently lose precision past $2^{53}$ in every language whose only number type is a double, with no warning at either conversion.⁵

The low-precision formats of machine learning

CS:APP covers the two IEEE-754 widths that existed when it was written, single and double. Machine learning has since driven a proliferation of narrower formats, and they are a direct application of everything above: each is the same sign-exponent-fraction layout, trading fraction bits (precision) against exponent bits (range) for a specific workload.

The IEEE half precision (fp16, 1/5/10) has a 5-bit exponent, so its range tops out near $65,504$ — small enough that gradients in a deep network overflow to infinity. Google's bfloat16 (1/8/7) keeps float's full 8-bit exponent and its range, spending only 7 bits on the fraction; it deliberately sacrifices precision to keep the dynamic range — the trade training tolerates and overflow does not.⁶ Training is far more sensitive to range (not overflowing) than to the last few bits of precision, so truncating a float to its top 16 bits — which is all bfloat16 is — costs almost nothing.

The trend has run all the way down to 8-bit floats, standardized by the OCP in 2023 in two flavors — E4M3 (more precision) and E5M2 (more range) — for the weights and activations of large models, where storing a number in one byte instead of four quarters the memory traffic that dominates inference cost.⁷ Each new format is the sign-exponent-fraction layout of this lesson, re-partitioned. Reading the low-precision literature comes down to two questions about any format: how many exponent bits (what is its range and where does it overflow), and how many fraction bits (how coarse is its spacing) — the same two knobs that separate float from double.

Floating point and integers cover the numbers; the final foundations lesson returns to the bits themselves as logical objects, in boolean algebra and bit manipulation.