---
title: Higher Derivatives, Taylor's Theorem, and Extrema
module: Functions of Several Variables (Introduction)
moduleNumber: 8
lessonNumber: 3
order: 803
summary: >
  Iterating the derivative gives a symmetric second derivative, the Hessian, whose
  mixed partials agree when they are continuous. Taylor's theorem expands a smooth
  map to any order with a Lagrange-type remainder, and at a critical point the
  definiteness of the Hessian decides between a local minimum, a local maximum, and
  a saddle.
topics: [Functions of Several Variables (Introduction)]
sources:
  - book: Shkoller
    ref: "§2.10 Higher-order derivatives and Cᵏ functions; §2.11 Taylor's theorem"
  - book: Shkoller
    ref: "§2.12 The minima and maxima of functions f : ℝⁿ → ℝ"
draft: false
---

The [derivative](/real-analysis/several-variables/differentiability-rn)
$Df : A \to L(\mathbb{R}^n, \mathbb{R}^m)$ is itself a map, from points to linear
transformations, and it can be differentiated again. Doing so once produces the
second derivative that carries the curvature of $f$; doing so repeatedly produces
the Taylor expansion. For a scalar field the second derivative is a symmetric
matrix, the Hessian, and its definiteness generalizes the sign condition of the
one-variable second-derivative test.

## The second derivative as a bilinear map

If $f : A \subseteq \mathbb{R}^n \to \mathbb{R}^m$ is differentiable, then
$Df : A \to L(\mathbb{R}^n, \mathbb{R}^m)$, where $L(\mathbb{R}^n, \mathbb{R}^m)$
is the space of linear maps (identified with $m \times n$ matrices). If this map
is itself differentiable at $x_0$, its derivative
$D^2 f(x_0) = D(Df)(x_0)$ sends a vector $u$ to a linear map, so that
$(D^2 f(x_0)\,u)\,v$ lies in $\mathbb{R}^m$ for each pair $u, v$. It is convenient
to read $D^2 f(x_0)$ as a **bilinear map**

$$
D^2 f(x_0)(u, v) = \bigl(D^2 f(x_0)\,u\bigr)\,v, \qquad u, v \in \mathbb{R}^n,
$$

linear in each argument separately. When $f$ is scalar-valued, the bilinear map is
represented by a single $n \times n$ matrix.

> **Theorem (Matrix of the second derivative).** For $f : A \subseteq \mathbb{R}^n
> \to \mathbb{R}$ twice differentiable at $x_0$, the matrix of $D^2 f(x_0)$ in the
> standard basis is
>
> $$
> D^2 f(x_0) =
> \begin{pmatrix}
> \dfrac{\partial^2 f}{\partial x_1 \partial x_1} & \cdots & \dfrac{\partial^2 f}{\partial x_1 \partial x_n} \\[2mm]
> \vdots & & \vdots \\[1mm]
> \dfrac{\partial^2 f}{\partial x_n \partial x_1} & \cdots & \dfrac{\partial^2 f}{\partial x_n \partial x_n}
> \end{pmatrix},
> $$
>
> each second-order partial evaluated at $x_0$. This is the **Hessian** of $f$.

The proof is immediate: $Df = (\partial f/\partial x_1, \dots, \partial f/\partial
x_n)$, and applying the Jacobian theorem from the first lesson to $Df$ produces the
matrix of second partials. Higher derivatives continue the pattern — $D^3 f(x_0)$
is a trilinear map with components $\partial^3 f / \partial x_i \partial x_j
\partial x_k$ — but the matrix picture stops at the second derivative of a scalar
field.

## Equality of mixed partials

The Hessian written above has, in general, an ordering ambiguity:
$\partial^2 f / \partial x_i \partial x_j$ differentiates first in $x_j$ then in
$x_i$, and swapping the order might change the answer. Under continuity of the
second partials, it does not.

> **Theorem (Symmetry of the Hessian, Clairaut).** Let $f : A \subseteq
> \mathbb{R}^n \to \mathbb{R}^m$ be twice differentiable on the open set $A$ with
> $D^2 f$ continuous. Then $D^2 f(x)$ is symmetric:
>
> $$
> \frac{\partial^2 f}{\partial x_i \partial x_j} = \frac{\partial^2 f}{\partial x_j \partial x_i}
> \qquad \text{for all } i, j.
> $$

> **Proof (two variables; the general case fixes the other coordinates).** For small
> $h, k$ form the second difference
>
> $$
> S_{h,k} = \bigl[ f(x+h, y+k) - f(x, y+k) \bigr] - \bigl[ f(x+h, y) - f(x, y) \bigr].
> $$
>
> Setting $g_k(u) = f(u, y+k) - f(u, y)$, we have $S_{h,k} = g_k(x+h) - g_k(x)$, and
> the [mean value theorem](/real-analysis/differentiation/mean-value-theorem) twice
> gives, for some intermediate points,
>
> $$
> S_{h,k} = \frac{\partial}{\partial y}\frac{\partial f}{\partial x}(c, d)\, hk.
> $$
>
> Because $S_{h,k}$ is symmetric under swapping the roles of $(h, x)$ and $(k, y)$,
> the same argument in the other order yields
> $S_{h,k} = \frac{\partial}{\partial x}\frac{\partial f}{\partial y}(\tilde c,
> \tilde d)\, hk$. Divide by $hk$ and let $(h, k) \to (0,0)$; continuity of the
> second partials collapses both to the same value at $(x, y)$. $\blacksquare$

> **Example (symmetry in a formula).** For $f(x, y) = y x^2 \cos(y^2)$,
> $$
> \frac{\partial^2 f}{\partial y \, \partial x} = 2x \cos(y^2) - 4 x y^2 \sin(y^2)
> = \frac{\partial^2 f}{\partial x \, \partial y},
> $$
> the two orders agreeing exactly, as the theorem guarantees for this smooth $f$.

**Smoothness classes.** A map whose first $k$ derivatives exist and are continuous
on $A$ is of class $C^k(A)$; if it is $C^k$ for every $k$ it is **smooth**, or
$C^\infty$. Symmetry of all higher derivatives follows by induction. The classes
nest, $C^\infty \subset \cdots \subset C^2 \subset C^1 \subset C^0$, and the
Hessian test below lives in $C^2$.

## Taylor's theorem in several variables

Taylor's theorem expands $f$ around a point as a polynomial in the displacement
plus a remainder controlled by the next derivative. The one-variable statement
generalizes with the $l$-th derivative acting as an $l$-linear form on $l$ copies
of the displacement.

> **Theorem (Taylor).** Let $f \in C^k(A)$ and let $x, y \in A$ with the segment
> joining them in $A$. Then there is a point $c$ on that segment with
>
> $$
> f(y) = f(x) + \sum_{l=1}^{k-1} \frac{1}{l!}\, D^l f(x)\,(y - x, \dots, y - x) + \frac{1}{k!}\, D^k f(c)\,(y - x, \dots, y - x),
> $$
>
> where $D^l f(x)(y-x, \dots, y-x)$ applies the $l$-linear map to $l$ copies of
> $y - x$. Setting $y = x + h$,
>
> $$
> f(x + h) = f(x) + Df(x)\,h + \tfrac{1}{2} D^2 f(x)(h, h) + \cdots + R_{k-1}(x, h),
> $$
>
> with remainder $R_{k-1}(x, h) / \lVert h \rVert^{k-1} \to 0$ as $h \to 0$.

The remainder has a Lagrange integral form,

$$
R_{k-1}(x, h) = \sum_{i_1, \dots, i_k} \int_0^1 \frac{(1 - t)^{k-1}}{(k-1)!}\, \frac{\partial^k f}{\partial x_{i_1} \cdots \partial x_{i_k}}(x + th)\, h_{i_1} \cdots h_{i_k}\,\d t,
$$

obtained by repeated integration by parts starting from the
[fundamental theorem of calculus](/real-analysis/riemann-integration/fundamental-theorem)
applied to $t \mapsto f(x + th)$. The case $k = 1$ is the mean value theorem; the
case $k = 2$ is the quadratic expansion the extremum test uses.

$$
% caption: The first-order Taylor term is the tangent line and the second-order
% term bends it into a parabola that hugs the graph more tightly near $x_0$.
\begin{tikzpicture}[>=stealth, font=\footnotesize, scale=1.0]
  \definecolor{acc}{HTML}{4A6FA5}
  \draw[black, ->] (-0.3,0) -- (6.2,0) node[right] {$x$};
  \draw[black, ->] (0,-0.3) -- (0,3.6);
  \def\xz{3.0}
  % f(x): a wiggle
  \draw[thick, black] plot[domain=0.4:5.7, samples=80] (\x, {1.7 + 0.55*sin(0.9*(\x-\xz) r)});
  \node[black, anchor=west] at (5.4,{1.7+0.55*sin(0.9*(5.7-\xz) r)}) {$f$};
  \pgfmathsetmacro\yz{1.7}
  \pgfmathsetmacro\slope{0.55*0.9}
  % tangent line (order 1)
  \draw[black, thick] (0.9,{\yz+\slope*(0.9-\xz)}) -- (5.1,{\yz+\slope*(5.1-\xz)});
  \node[black, anchor=north] at (1.35,0.5) {order 1};
  % order-2 parabola: y = yz + slope*(x-xz) - 0.5*0.55*0.81*(x-xz)^2
  \draw[acc, thick] plot[domain=1.0:5.0, samples=60] (\x, {\yz + \slope*(\x-\xz) - 0.5*0.55*0.81*(\x-\xz)*(\x-\xz)});
  \node[acc, anchor=north] at (4.6,{\yz + \slope*(4.6-\xz) - 0.5*0.55*0.81*(4.6-\xz)*(4.6-\xz)}) {order 2};
  \fill[black] (\xz,\yz) circle (1.8pt);
  \node[black, anchor=north west] at (\xz,\yz) {$x_0$};
\end{tikzpicture}
$$

**Real analytic functions.** Letting $k \to \infty$ gives the Taylor series
$\sum_l \frac{1}{l!} D^l f(x_0)(x - x_0, \dots)$. When it converges to $f$ near
$x_0$, $f$ is **real analytic** there — equivalent to the remainder tending to
$0$. Convergence is not automatic even for smooth functions.

> **Example (smooth but not analytic).** The function $f(x) = e^{-1/x}$ for
> $x > 0$ and $f(x) = 0$ for $x \le 0$ is $C^\infty(\mathbb{R})$: every derivative
> at $0$ equals $0$ (by l'Hôpital after $y = 1/x$). Its Taylor series about $0$ is
> identically zero, which does not equal $f$ for $x > 0$. So $f$ is smooth but not
> real analytic.

$$
% caption: The bump $f(x)=e^{-1/x}$ (and $0$ for $x\le 0$) lifts off the axis so
% flatly that every derivative at the origin is zero, yet $f>0$ just to the right.
\begin{tikzpicture}[>=stealth, font=\footnotesize, scale=1.0]
  \definecolor{acc}{HTML}{4A6FA5}
  \draw[black, ->] (-2.4,0) -- (3.8,0) node[right] {$x$};
  \draw[black, ->] (0,-0.3) -- (0,2.6) node[above] {$y$};
  \draw[thick, acc] (-2.2,0) -- (0,0);
  \draw[thick, acc] plot[domain=0.06:3.5, samples=80] (\x, {3*exp(-1/\x)});
  \node[acc, anchor=south east] at (3.5,{3*exp(-1/3.5)}) {$y = f(x)$};
  \fill[black] (0,0) circle (1.8pt);
  \node[black, anchor=north, align=center] at (0.3,-0.35) {every derivative\\ vanishes here};
\end{tikzpicture}
$$

> **Example (second-order expansion).** For $f(x, y) = \sin(x + 2y)$ near
> $(0, 0)$: $f(0,0) = 0$, the first partials are $(1, 2)$, and all second partials
> vanish at the origin. Thus $f(h, k) = h + 2k + R_2(h, k)$ with
> $R_2(h,k)/\lVert (h,k) \rVert^2 \to 0$.

## Critical points and extrema

Taylor's second-order expansion turns the search for local maxima and minima into
linear algebra. First, the first-order condition.

> **Definition (Critical point).** A point $x_0 \in A$ is a **local maximum** of
> $f$ if $f(x_0) \ge f(x)$ for $x$ in some neighborhood, a **local minimum** if
> $f(x_0) \le f(x)$ there; either is an **extreme** point. A point is a
> **critical point** if $f$ is differentiable there and $Df(x_0) = 0$.

> **Theorem (First-order condition).** If $f : A \subseteq \mathbb{R}^n \to
> \mathbb{R}$ is differentiable, $A$ open, and $x_0$ is an extreme point, then
> $Df(x_0) = 0$.

> **Proof.** If $Df(x_0) \neq 0$, pick $x$ with $Df(x_0)\,x = c > 0$. For small
> $\lambda > 0$, the definition of the derivative gives
> $f(x_0 + \lambda x) - f(x_0) \ge \lambda c / 2 > 0$ and
> $f(x_0 - \lambda x) - f(x_0) \le -\lambda c / 2 < 0$, so $x_0$ is neither a maximum
> nor a minimum. $\blacksquare$

A critical point need not be extreme. In one variable $f(x) = x^3$ has $f'(0) = 0$
but no extremum; in two variables $f(x, y) = y^2 - x^2$ has $Df = (-2x, 2y)$
vanishing at the origin, yet every neighborhood of $(0,0)$ contains points where
$f > 0$ and points where $f < 0$. Such a critical point is a **saddle point**.

$$
% caption: A saddle: along one principal axis the value rises, along the other it
% falls, so the critical point at the center is neither a maximum nor a minimum.
\begin{tikzpicture}[>=stealth, font=\footnotesize, scale=1.0]
  \definecolor{acc}{HTML}{4A6FA5}
  % hyperbolic level curves as bezier arcs (right/left/top/bottom branch pairs)
  \foreach \d in {0.7,1.35,2.05} {
    % right and left branches
    \draw[black] (\d,1.9) .. controls ({\d*0.72},0.6) and ({\d*0.72},-0.6) .. (\d,-1.9);
    \draw[black] (-\d,1.9) .. controls ({-\d*0.72},0.6) and ({-\d*0.72},-0.6) .. (-\d,-1.9);
    % top and bottom branches
    \draw[black] (1.9,\d) .. controls (0.6,{\d*0.72}) and (-0.6,{\d*0.72}) .. (-1.9,\d);
    \draw[black] (1.9,-\d) .. controls (0.6,{-\d*0.72}) and (-0.6,{-\d*0.72}) .. (-1.9,-\d);
  }
  % principal axes with rise/fall arrows
  \draw[acc, very thick, ->] (0,0) -- (2.3,0);
  \draw[acc, very thick, ->] (0,0) -- (-2.3,0);
  \node[acc, anchor=south west] at (2.0,0) {value rises};
  \draw[black, very thick, ->] (0,0) -- (0,2.3);
  \draw[black, very thick, ->] (0,0) -- (0,-2.3);
  \node[black, anchor=south east] at (0,2.2) {value falls};
  \fill[black] (0,0) circle (2.2pt);
  \node[black, anchor=north west] at (0.08,-0.05) {saddle};
\end{tikzpicture}
$$

**The Hessian test.** The second-order term of Taylor's expansion at a critical
point is $\tfrac{1}{2} D^2 f(x_0)(h, h)$, and its sign for all small $h$ decides
the classification.

> **Definition (Definiteness).** A symmetric bilinear form $B$ on $\mathbb{R}^n$
> is **positive definite** if $B(v, v) > 0$ for all $v \neq 0$, **negative
> definite** if $B(v, v) < 0$ for all $v \neq 0$, and **semidefinite** if the
> corresponding weak inequality holds.

> **Theorem (Second-order test).** Let $f \in C^2(A)$ and let $x_0$ be a critical
> point.
> - If $D^2 f(x_0)$ is negative definite, $x_0$ is a local maximum.
> - If $D^2 f(x_0)$ is positive definite, $x_0$ is a local minimum.
> - Conversely, a local maximum forces $D^2 f(x_0)$ negative **semi**definite, and
>   a local minimum forces it positive semidefinite.

> **Proof (minimum case).** If $D^2 f(x_0)$ is positive definite, then by continuity
> of the bilinear form on the compact unit sphere there is $\alpha > 0$ with
> $D^2 f(x_0)(v, v) \ge \alpha \lVert v \rVert^2$. Continuity of $D^2 f$ keeps
> $D^2 f(c)$ close to $D^2 f(x_0)$ for $c$ near $x_0$, so Taylor's expansion
> $f(y) - f(x_0) = \tfrac{1}{2} D^2 f(c)(y - x_0, y - x_0)$ (the first-order term
> vanishes) stays $\ge \tfrac{\alpha}{4} \lVert y - x_0 \rVert^2 > 0$ nearby. Hence a
> strict local minimum. $\blacksquare$

The gap between "definite" (sufficient) and "semidefinite" (necessary) is real:
$f(x, y) = x^2$ has a minimum along the $y = 0$ line where the Hessian is only
semidefinite, since it has zero curvature in the $y$-direction.

$$
% caption: The Hessian's curvature signs at a critical point: curving up in every
% direction gives a minimum, down a maximum, and up in one and down in another a saddle.
\begin{tikzpicture}[>=stealth, font=\footnotesize, scale=1.0]
  \definecolor{acc}{HTML}{4A6FA5}
  % panel 1: minimum (bowl)
  \begin{scope}[xshift=0cm]
    \draw[black] (-1.1,0) parabola bend (0,-0.9) (1.1,0);
    \draw[acc, thick] (-1.1,0.1) parabola bend (0,-0.75) (1.1,0.1);
    \node[align=center, anchor=north] at (0,-1.15) {both curve up\\ local minimum};
  \end{scope}
  % panel 2: maximum (dome)
  \begin{scope}[xshift=4cm]
    \draw[black] (-1.1,-0.9) parabola bend (0,0) (1.1,-0.9);
    \draw[acc, thick] (-1.1,-1.0) parabola bend (0,-0.15) (1.1,-1.0);
    \node[align=center, anchor=north] at (0,-1.15) {both curve down\\ local maximum};
  \end{scope}
  % panel 3: saddle
  \begin{scope}[xshift=8cm]
    \draw[acc, thick] (-1.1,-0.75) parabola bend (0,0.05) (1.1,-0.75);
    \draw[black, thick] (-1.1,0.85) parabola bend (0,0.05) (1.1,0.85);
    \node[align=center, anchor=north] at (0,-1.15) {one up, one down\\ saddle};
  \end{scope}
\end{tikzpicture}
$$

**The $2 \times 2$ criterion.** For $f : \mathbb{R}^2 \to \mathbb{R}$ the Hessian
is $\left(\begin{smallmatrix} a & b \\ b & d \end{smallmatrix}\right)$ with
$a = f_{xx}$, $b = f_{xy}$, $d = f_{yy}$, and definiteness reduces to two scalar
checks.

- **Positive definite** iff $a > 0$ and $ad - b^2 > 0$: local minimum.
- **Negative definite** iff $a < 0$ and $ad - b^2 > 0$: local maximum.
- **$ad - b^2 < 0$**: indefinite, a saddle.

> **Worked example (a minimum).** For $f(x, y) = x^2 - xy + y^2$, the critical point
> solves $2x - y = 0$ and $-x + 2y = 0$, giving $(0,0)$. The Hessian is
> $\left(\begin{smallmatrix} 2 & -1 \\ -1 & 2 \end{smallmatrix}\right)$ with
> $a = 2 > 0$ and $ad - b^2 = 3 > 0$, so $(0,0)$ is a local minimum. The eigenvalues
> $1$ and $3$ are the principal curvatures, both positive.

> **Worked example (a minimum and a saddle).** For $f(x, y) = x^3 - 3x^2 + y^2$,
> $\partial f/\partial x = 3x^2 - 6x$ and $\partial f/\partial y = 2y$ vanish at
> $(0,0)$ and $(2,0)$. The Hessian is
> $\left(\begin{smallmatrix} 6x - 6 & 0 \\ 0 & 2 \end{smallmatrix}\right)$. At
> $(2, 0)$ it is $\left(\begin{smallmatrix} 6 & 0 \\ 0 & 2 \end{smallmatrix}\right)$,
> positive definite, a local minimum; at $(0, 0)$ it is
> $\left(\begin{smallmatrix} -6 & 0 \\ 0 & 2 \end{smallmatrix}\right)$, indefinite, a
> saddle.

$$
% caption: The two critical points of $x^3-3x^2+y^2$: the origin is a saddle (value
% rises in $y$, falls in $x$); $(2,0)$ is a minimum (value rises in every direction).
\begin{tikzpicture}[>=stealth, font=\footnotesize, scale=1.0]
  \definecolor{acc}{HTML}{4A6FA5}
  \draw[black, ->] (-1.4,0) -- (3.8,0) node[right] {$x$};
  \draw[black, ->] (0,-1.7) -- (0,1.9) node[above] {$y$};
  % saddle at origin
  \fill[black] (0,0) circle (2pt);
  \node[black, anchor=south east] at (-0.05,0.08) {saddle};
  \draw[->, black, thick] (0,0) -- (0,1.1);
  \draw[->, black, thick] (0,0) -- (0,-1.1);
  \node[black, anchor=west] at (0.08,0.95) {rises};
  \draw[->, black, thick] (0,0) -- (-0.8,0);
  \draw[->, black, thick] (0,0) -- (0.8,0);
  \node[black, anchor=north] at (-0.55,-0.05) {falls};
  % minimum at (2,0)
  \fill[acc] (2,0) circle (2pt);
  \node[acc, anchor=west] at (2.14,0.55) {min};
  \draw[->, acc, thick] (2,0) -- (2,0.95);
  \draw[->, acc, thick] (2,0) -- (2,-0.95);
  \draw[->, acc, thick] (2,0) -- (2.8,0);
  \draw[->, acc, thick] (2,0) -- (1.2,0);
  \node[acc, anchor=west] at (3.0,0.6) {rises};
\end{tikzpicture}
$$

| Hessian at critical point | $2 \times 2$ signs | Classification |
| --- | --- | --- |
| positive definite | $a > 0,\ ad - b^2 > 0$ | local minimum |
| negative definite | $a < 0,\ ad - b^2 > 0$ | local maximum |
| indefinite | $ad - b^2 < 0$ | saddle |
| semidefinite (a zero eigenvalue) | $ad - b^2 = 0$ | test inconclusive |

The inconclusive row is where the second-order expansion runs out: the quadratic
form has a flat direction, and deciding the extremum needs higher-order terms or a
direct argument. Local invertibility of $f$ itself is governed not by
definiteness of the Hessian but by nonsingularity of the Jacobian, through the
[inverse and implicit function theorems](/real-analysis/several-variables/inverse-implicit-theorems).[^sh-taylor]

[^sh-taylor]: **Shkoller**, _MAT125B Lecture Notes_, §2.10 (second derivative as a bilinear map, Hessian matrix Theorem 2.43, symmetry Theorem 2.44), §2.11 (Taylor's theorem 2.50 with Lagrange remainder; Examples 2.51–2.53), and §2.12 (critical points Theorem 2.56, definiteness test Theorem 2.60, the $2\times 2$ Lemmas 2.61–2.62, Examples 2.63–2.64).