---
title: Nonstandard Analysis
module: Models, Compactness, and Theories
moduleNumber: 5
lessonNumber: 4
order: 504
summary: >
  Compactness builds a model of the real ordered field containing infinite
  elements and nonzero infinitesimals. The transfer principle carries every
  first-order truth from the reals to this extension, the standard-part map
  collapses finite hyperreals back onto the reals, and continuity and the
  derivative are rederived by working with infinitely small quantities directly.
topics: [Models, Compactness, and Theories]
sources:
  - book: Enderton
    ref: "Ch. 2 — First-Order Logic; §2.8 Nonstandard Analysis"
draft: false
---

Leibniz and Newton built the calculus on quantities infinitely small yet
nonzero. Newton's $o$ could be multiplied by any finite number and stay
negligible, but had to be nonzero so one could divide by it; Leibniz's $\d x$ was
smaller than any assignable quantity, again nonzero. The idea drew attacks
through the eighteenth century, and the nineteenth century replaced it with the
$\varepsilon$-$\delta$ treatment of limits now standard. In 1961 Abraham
Robinson recovered infinitesimals rigorously by working inside a nonstandard
model of the theory of the reals. The tools are compactness, to build the model,
and elementary equivalence, to transfer truths into it.

## Building the hyperreal field

The construction uses a deliberately enormous first-order language: one that can
name every feature of $\mathbb{R}$. Its parameters are

- $\forall$, ranging over the reals;
- an $n$-place predicate symbol $P_R$ for _every_ $n$-ary relation $R$ on
  $\mathbb{R}$;
- a constant symbol $c_r$ for every real $r$;
- an $n$-place function symbol $f_F$ for _every_ $n$-ary operation $F$ on
  $\mathbb{R}$.

The standard structure $\fR$ interprets each symbol by the object it
names. A nonstandard structure comes from compactness. Let

$$
\Gamma = \Th \fR \cup \{\, c_r \mathbin{P_<} v_1 \mid r \in \mathbb{R} \,\},
$$

where $c_r \mathbin{P_<} v_1$ formalizes "$r$ is less than $v_1$." Any finite
subset of $\Gamma$ names finitely many reals below $v_1$, and is satisfied in
$\fR$ by assigning $v_1$ a large enough real. By compactness $\Gamma$
has a model $\fA$ with an element $a$ satisfying every
$c_r \mathbin{P_<} v_1$. Since $\fA \models \Th \fR$, we have
$\fA \equiv \fR$.

The map $h(r) = c_r^{\fA}$ embeds $\mathbb{R}$ into $\fA$. It is
injective and preserves every relation and operation, because each such
preservation is a sentence true in $\fR$ and hence in the elementarily
equivalent $\fA$. Replacing the image points by the reals themselves
yields a structure ${}^\ast\mathbb{R}$ containing $\mathbb{R}$ as a
substructure, with an element $b$ larger than every real.[^constr]

$$
% caption: Compactness forces an element $b$ above every standard real; its
% reciprocal is a nonzero infinitesimal, and $\mathbb{R}$ sits inside the
% extension as a substructure.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \draw[thick] (-0.2,0) -- (9.2,0);
  \foreach \x/\n in {0.6/0, 2.0/1, 3.4/2} {\fill (\x,0) circle (1.6pt); \node[below, font=\scriptsize] at (\x,-0.12) {\n};}
  \foreach \x in {4.4,4.8,5.2} \fill (\x,0) circle (0.8pt);
  \draw[thick] (0.6,0.45) -- (0.6,0.62) -- (3.4,0.62) -- (3.4,0.45);
  \node[font=\scriptsize] at (2.0,1.0) {standard reals};
  \fill[acc] (8.0,0) circle (2.2pt);
  \node[acc, below, font=\scriptsize] at (8.0,-0.12) {b};
  \node[acc, align=center, font=\scriptsize] at (8.0,0.95) {inf\/inite element\\above every real};
\end{tikzpicture}
$$

Notation follows the reals with an asterisk: ${}^\ast R$ is the relation, and
${}^\ast F$ the operation, that ${}^\ast\mathbb{R}$ assigns to $P_R$ and $f_F$.
Each standard relation is the restriction of its starred version to
$\mathbb{R}$.

## The transfer principle

The value of $\fA \equiv \fR$ is a general method for proving
facts about the extension. To show that a starred relation or operation has a
property, observe that (1) the standard relation or operation has it, (2) the
property is expressible by a sentence of the language, and (3)
$\fR \equiv {}^\ast\mathbb{R}$. The property then transfers.

> **Transfer principle.** A first-order sentence of the language holds in
> $\mathbb{R}$ if and only if it holds in ${}^\ast\mathbb{R}$.

$$
% caption: Any property expressible as a first-order sentence carries between the
% reals and the hyperreals in both directions, because the two structures are
% elementarily equivalent.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \node[draw, minimum width=26mm, minimum height=10mm, align=center] (r) at (0,0) {true in the reals};
  \node[draw, minimum width=30mm, minimum height=10mm, align=center] (hr) at (6.2,0) {true in the hyperreals};
  \draw[->, acc, thick] (r.north east) .. controls (3.1,1.1) .. (hr.north west);
  \draw[->, acc, thick] (hr.south west) .. controls (3.1,-1.1) .. (r.south east);
  \node[acc, font=\scriptsize] at (3.1,1.0) {same sentence};
  \node[acc, font=\scriptsize] at (3.1,-1.0) {same sentence};
\end{tikzpicture}
$$

Transfer proves at once that ${}^\ast<$ is a linear order, that ${}^\ast+$ is
commutative, and that $({}^\ast\mathbb{R}; 0, 1, {}^\ast+, {}^\ast\cdot)$ is a
field, since each field axiom is a single sentence true in $\mathbb{R}$.
Properties _not_ expressible by a sentence can fail. The
least-upper-bound property is one: $\mathbb{R}$ is a bounded subset of
${}^\ast\mathbb{R}$, bounded by any infinite $b$, yet has no least upper bound
there. The least-upper-bound property quantifies over subsets, which
first-order sentences cannot reach.

> **Worked example (Transfer of transitivity by the three-step method).**
> To see that ${}^\ast<$ is transitive, run the general method. (1) The standard
> $<$ is transitive on $\mathbb{R}$. (2) Transitivity is a single sentence of the
> language,
> $$
> \forall x \, \forall y \, \forall z \,(x \mathbin{P_<} y \to y \mathbin{P_<} z \to x \mathbin{P_<} z),
> $$
> true in $\fR$. (3) Since $\fR \equiv {}^\ast\mathbb{R}$, the same sentence holds
> in ${}^\ast\mathbb{R}$, where $P_<$ is interpreted by ${}^\ast<$. So
> ${}^\ast<$ is transitive. Trichotomy transfers the same way, making
> ${}^\ast<$ a linear order, and each field axiom transfers to make
> $({}^\ast\mathbb{R}; 0, 1, {}^\ast+, {}^\ast\cdot)$ a field.

## Finite elements and infinitesimals

The extension stratifies by magnitude. Write ${}^\ast|x|$ for the starred
absolute value.

- **Finite elements.** $\mathbb{F} = \{x \in {}^\ast\mathbb{R} \mid
  {}^\ast|x| < y \text{ for some } y \in \mathbb{R}\}$: bounded by a standard
  real.
- **Infinitesimals.** $\mathbb{I} = \{x \in {}^\ast\mathbb{R} \mid
  {}^\ast|x| < y \text{ for every positive } y \in \mathbb{R}\}$: below every
  positive standard real.

The reciprocal of the infinite $b$ is a nonzero infinitesimal, so $\mathbb{I}$
contains more than $0$. The only _standard_ infinitesimal is $0$ itself.
Unbounded standard sets acquire infinite points: ${}^\ast\mathbb{N}$ contains
infinite natural numbers, since "for every real there is a larger member of
$\mathbb{N}$" transfers.

$$
% caption: The finite hyperreals cluster into monads around each real; beyond
% them lie the infinite elements, and inside each monad the differences from its
% center are infinitesimal.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \draw[thick] (-0.2,0) -- (8.6,0);
  \foreach \x/\n in {1.0/0, 3.0/1, 5.0/2} {\fill (\x,0) circle (1.7pt); \node[below, font=\scriptsize] at (\x,-0.14) {\n};}
  \foreach \x in {1.0,3.0,5.0} {
    \foreach \d in {-0.22,-0.11,0.11,0.22} \fill[acc] (\x+\d,0) circle (0.7pt);
  }
  \foreach \x in {6.0,6.3,6.6} \fill (\x,0) circle (0.8pt);
  \node[acc, font=\scriptsize, align=center] at (3.0,1.35) {monad: real plus its inf\/initesimals};
  \draw[acc, thick] (3.0,1.05) -- (3.0,0.35);
  \node[font=\scriptsize, align=center] at (7.7,0.8) {inf\/inite\\region};
  \draw[thick] (6.9,0.35) -- (6.9,0.52) -- (8.5,0.52) -- (8.5,0.35);
\end{tikzpicture}
$$

> **Theorem (Arithmetic of finite and infinitesimal elements).** (a) $\mathbb{F}$ is closed under addition, subtraction, and
> multiplication. (b) $\mathbb{I}$ is closed under addition and subtraction, and
> under multiplication by any finite element: $x \in \mathbb{I}$ and
> $z \in \mathbb{F}$ give $x \cdot z \in \mathbb{I}$.

In algebraic language $\mathbb{F}$ is a subring of ${}^\ast\mathbb{R}$ and
$\mathbb{I}$ is an ideal of $\mathbb{F}$. The proofs are direct bounds: if
${}^\ast|x| < a$ and ${}^\ast|y| < b$ for standard $a, b$, then
${}^\ast|x \cdot y| < a \cdot b$, a standard real, so the product is finite; and
if $x$ is infinitesimal while $z$ is finite with ${}^\ast|z| < b$, then for any
positive standard $a$ one has ${}^\ast|x| < a/b$, whence
${}^\ast|x \cdot z| < a$.

## The standard part

Infinitesimal closeness organizes $\mathbb{F}$ into a copy of $\mathbb{R}$.

> **Definition (Infinitely close).** $x \simeq y$ iff $x - y$ is infinitesimal.

By the arithmetic of finite and infinitesimal elements this is an equivalence relation, compatible with addition and
(on finite elements) multiplication. Distinct standard reals are never
infinitely close, since $0$ is the only standard infinitesimal. Every finite
hyperreal is infinitely close to exactly one real.

> **Theorem (Existence of the standard part).** Every $x \in \mathbb{F}$ is infinitely close to a unique
> $r \in \mathbb{R}$.

> **Proof.** Let $r$ be the least upper bound of the standard reals below $x$,
> which exists in $\mathbb{R}$ because $x$ is finite. If $x \not\simeq r$, then
> some standard $q$ lies strictly between them, and $q$ is either an upper bound of
> the set violating leastness, or below $x$ violating the bound — a contradiction.
> So $x \simeq r$. Uniqueness holds because two reals infinitely close to $x$ are
> infinitely close to each other, hence equal. $\square$

> **Definition (Standard part).** For finite $x$, the unique real $r \simeq x$
> is the **standard part** $\mathrm{st}(x)$. Each finite $x$ decomposes uniquely
> as $x = \mathrm{st}(x) + i$ with $i$ infinitesimal.

$$
% caption: The standard-part map sends every finite hyperreal to the one real it
% surrounds, collapsing each monad to its center.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \foreach \d/\i in {-0.35/1,-0.15/2,0.05/3,0.28/4} \fill[acc] (3.0+\d,1.3) circle (1.1pt);
  \node[acc, font=\scriptsize] at (3.0,1.85) {f\/inite hyperreals near a real};
  \draw[thick] (0.5,0) -- (5.5,0);
  \fill (3.0,0) circle (2.0pt);
  \node[below, font=\scriptsize] at (3.0,-0.16) {r};
  \foreach \ds/\de/\ye in {-0.35/-0.20/0.44, -0.15/-0.07/0.50, 0.05/0.07/0.50, 0.28/0.20/0.44}
    \draw[->, acc] (3.0+\ds,1.15) -- (3.0+\de,\ye);
  \node[acc, font=\scriptsize, anchor=west] at (3.42,0.8) {st};
\end{tikzpicture}
$$

> **Theorem (The standard-part homomorphism).** The map $\mathrm{st}$ sends $\mathbb{F}$ onto $\mathbb{R}$,
> vanishes exactly on the infinitesimals, and respects sums and products:
> $\mathrm{st}(x + y) = \mathrm{st}(x) + \mathrm{st}(y)$ and
> $\mathrm{st}(x \cdot y) = \mathrm{st}(x) \cdot \mathrm{st}(y)$.

So $\mathrm{st}$ is a ring homomorphism from $\mathbb{F}$ onto $\mathbb{R}$ with
kernel $\mathbb{I}$, and the quotient $\mathbb{F}/\mathbb{I}$ is isomorphic to
the real field.[^alg] From here the asterisks on the arithmetic operations are
dropped.

> **Worked example (Standard part of a finite hyperreal).** Let $b$ be the
> infinite element above every real, so its reciprocal $1/b$ is a nonzero
> infinitesimal. The hyperreal $x = 2 + 1/b$ is finite, bounded by the standard
> $3$. Its standard part is
> $$
> \mathrm{st}(x) = \mathrm{st}(2) + \mathrm{st}(1/b) = 2 + 0 = 2,
> $$
> using that $\mathrm{st}$ respects sums and vanishes on infinitesimals. The
> decomposition guaranteed by the theorem reads $x = \mathrm{st}(x) + i$ with
> $i = 1/b$ infinitesimal. No other real is infinitely close to $x$, since
> distinct standard reals are never infinitely close, so $2$ is the unique real
> that $x$ surrounds.

## Convergence and the derivative

The infinitesimal apparatus rephrases limits without $\varepsilon$-$\delta$
quantifiers. Instead of variables approaching a value, variables land infinitely
close to it.

> **Definition (Convergence).** For $F : \mathbb{R} \to \mathbb{R}$, $F$
> converges at $a$ to $b$ iff whenever $x$ is infinitely close to $a$ but
> different from it, ${}^\ast F(x)$ is infinitely close to $b$.

This matches the classical definition. If $F$ converges to $b$ in the ordinary
sense, then for each standard $\varepsilon > 0$ the sentence guaranteeing a
$\delta$ transfers to ${}^\ast\mathbb{R}$, and an $x$ infinitely close to $a$
satisfies ${}^\ast|x - a| < \delta$, forcing ${}^\ast|b - {}^\ast F(x)| <
\varepsilon$ for every standard $\varepsilon$, so ${}^\ast F(x) \simeq b$.
Conversely, if the infinitesimal condition holds, then for each standard
$\varepsilon > 0$ a suitable $\delta$ exists in ${}^\ast\mathbb{R}$ (take it
infinitesimal), and the sentence transfers back to $\mathbb{R}$. The limit is
then the standard part of a single evaluation:

$$
\lim_{x \to a} F(x) = \mathrm{st}\big({}^\ast F(a + i)\big) \quad \text{for any nonzero infinitesimal } i.
$$

> **Corollary (Nonstandard criterion for continuity).** $F$ is continuous at $a$ iff $x \simeq a$ implies
> ${}^\ast F(x) \simeq F(a)$.

The derivative becomes an ordinary quotient. Writing $\d F = {}^\ast F(a + \d x) -
F(a)$ for a nonzero infinitesimal $\d x$,

$$
F'(a) = \mathrm{st}\!\left(\frac{\d F}{\d x}\right),
$$

and the division is genuine division in the field, not a limit of ratios.

$$
% caption: The derivative is the standard part of the difference quotient over a
% single nonzero infinitesimal $dx$, an ordinary field division rather than a
% limiting process.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \node[draw, minimum width=24mm, minimum height=11mm, align=center] (q) at (0,0) {$\frac{dF}{dx}$};
  \node[draw=acc, thick, minimum width=24mm, minimum height=11mm, align=center] (d) at (5.4,0) {derivative at a};
  \draw[->, acc, thick] (q) -- (d);
  \node[acc, font=\scriptsize] at (2.7,0.4) {standard part};
  \node[font=\scriptsize, align=center] at (0,-1.2) {inf\/initesimal step $dx$};
\end{tikzpicture}
$$

> **Worked example (Derivative of $x^2$ by an infinitesimal).** For
> $F(x) = x^2$ and any nonzero infinitesimal $\d x$,
> $$
> \frac{\d F}{\d x} = \frac{(a + \d x)^2 - a^2}{\d x} = \frac{2a \,\d x + (\d x)^2}{\d x} = 2a + \d x \simeq 2a,
> $$
> so $F'(a) = \mathrm{st}(2a + \d x) = 2a$. No limit is taken; the infinitesimal
> $\d x$ is added algebraically and then discarded by the standard part.

The classical theorems follow with nonstandard proofs. Differentiability at $a$
implies continuity at $a$: if $\d F/\d x \simeq F'(a)$ is finite, multiplying by the
infinitesimal $\d x$ shows $\d F$ is infinitesimal, so ${}^\ast F(a + \d x) \simeq
F(a)$. The chain rule reads off as a product of quotients:

$$
\frac{\d F}{\d x} = \frac{\d F}{\d G} \cdot \frac{\d G}{\d x} \simeq F'(G(a)) \cdot G'(a),
$$

where $\d G = {}^\ast G(a + \d x) - G(a)$, handling the case $\d G = 0$ separately.
These are classical theorems with nonstandard proofs, not analogues of them; the
method has produced original results in analysis, including in the theory of
Hilbert spaces.[^calc]

| Classical notion | Nonstandard rendering |
| --- | --- |
| $\lim_{x\to a} F(x) = b$ | $x \simeq a$, $x \neq a$ gives ${}^\ast F(x) \simeq b$ |
| continuity at $a$ | $x \simeq a$ gives ${}^\ast F(x) \simeq F(a)$ |
| derivative $F'(a)$ | $\mathrm{st}(\d F/\d x)$ for infinitesimal $\d x$ |
| $\varepsilon$-$\delta$ argument | transfer of a first-order sentence |

Three tools combine in the construction: compactness supplies the
[infinite element](/logic/models-and-theories/compactness-and-lowenheim-skolem),
elementary equivalence supplies the
[transfer principle](/logic/models-and-theories/interpretations-between-theories)
carrying every first-order truth, and the arithmetic of infinitesimals does
the rest. What separates ${}^\ast\mathbb{R}$ from $\mathbb{R}$ is precisely what
first-order logic cannot express: the least-upper-bound property and the
distinction between finite and infinite elements.

[^constr]: Enderton, §2.8 — construction of ${}^\ast\mathbb{R}$ by compactness, the embedding $h$, and the general transfer method from $\fR \equiv {}^\ast\mathbb{R}$.
[^alg]: Enderton, §2.8 — finite elements and infinitesimals (Theorem 28A), infinite closeness, and the standard-part homomorphism (Theorems 28D, 28F).
[^calc]: Enderton, §2.8 — convergence, continuity, and the derivative in infinitesimal terms (Corollary 28G, the chain rule).