---
title: Quadratic Forms
module: Symmetric Matrices, Quadratic Forms, and the SVD
moduleNumber: 7
lessonNumber: 2
order: 702
summary: >
  A quadratic form xᵀAx is the second-degree analogue of a linear map, attached
  to a symmetric matrix A. Orthogonal diagonalization changes variables to the
  eigenbasis, removing all cross-terms and rotating the form into standard
  position. The signs of the eigenvalues then classify it as definite or
  indefinite.
topics: [Symmetric Matrices, Quadratic Forms, and the SVD]
sources:
  - book: Lay
    ref: "Ch. 7 — Symmetric Matrices and Quadratic Forms; §7.2 Quadratic Forms"
draft: false
---

Linear systems involve expressions of degree one in the variables. The next
step up is degree two. Sums of squares already appeared in the
[norm](/linear-algebra/orthogonality-least-squares/inner-product-length-orthogonality)
$\|x\|^2 = x^\top x$; a general second-degree homogeneous expression, allowing
cross-products $x_i x_j$, is a quadratic form. These occur throughout
applications: as energies in physics, as the objective in optimization, as the
noise power of a signal, and as the covariance ellipsoids of statistics.
[Symmetric-matrix theory](/linear-algebra/symmetric-quadratic-svd/diagonalizing-symmetric-matrices)
supplies the tools to analyze them.

> **Definition (Quadratic form).** A function $Q : \mathbb{R}^n \to \mathbb{R}$
> of the form
> $$
> Q(x) = x^\top A x,
> $$
> where $A$ is an $n \times n$ symmetric matrix, called the **matrix of the
> quadratic form**.

The simplest nonzero form is $Q(x) = x^\top I x = \|x\|^2$. In general, $x^\top
A x$ is a scalar assembled from products of the coordinates, with the diagonal
of $A$ supplying the squared terms and the off-diagonal entries the
cross-terms.

## From matrix to form and back

Multiplying out $x^\top A x$ for a $2 \times 2$ matrix shows how the entries
enter. With $x = (x_1, x_2)$,

$$
\begin{bmatrix} x_1 & x_2 \end{bmatrix}
\begin{bmatrix} 3 & -2 \\ -2 & 7 \end{bmatrix}
\begin{bmatrix} x_1 \\ x_2 \end{bmatrix}
= 3x_1^2 - 4x_1 x_2 + 7x_2^2.
$$

The squared coefficients $3$ and $7$ come from the diagonal. The cross-term
$-4x_1 x_2$ collects both off-diagonal entries: the $-2$ in position $(1,2)$ and
the matching $-2$ in position $(2,1)$ each contribute $-2 x_1 x_2$. A diagonal
matrix produces no cross-term at all.

Reversing the direction, any quadratic form has a unique **symmetric** matrix.
The rule: the coefficient of $x_i^2$ goes on the diagonal, and the coefficient
of $x_i x_j$ (for $i \neq j$) is split evenly between positions $(i,j)$ and
$(j,i)$. For the three-variable form
$Q(x) = 5x_1^2 + 3x_2^2 + 2x_3^2 - x_1 x_2 + 8 x_2 x_3$,

$$
A = \begin{bmatrix} 5 & -1/2 & 0 \\ -1/2 & 3 & 4 \\ 0 & 4 & 2 \end{bmatrix}.
$$

The $x_1 x_2$ coefficient $-1$ splits into two $-\tfrac12$ entries; the $x_2 x_3$
coefficient $8$ splits into two $4$ entries; the absent $x_1 x_3$ term leaves
zeros in positions $(1,3)$ and $(3,1)$. Splitting evenly is what keeps $A$
symmetric, and symmetry is what makes the Spectral Theorem available.

## Removing the cross-terms

Cross-terms make a quadratic form awkward to read: the sign of $Q$ and its
extreme values are obscured by the coupling between variables. A change of
variable removes them.

> **Definition (Change of variable).** For an invertible matrix $P$, the
> substitution $x = Py$ (equivalently $y = P^{-1}x$) re-expresses $x$ in the
> coordinates $y$ relative to the basis given by the columns of $P$.

Substituting $x = Py$ into $x^\top A x$ and collecting the matrix,

$$
x^\top A x = (Py)^\top A (Py) = y^\top (P^\top A P)\, y,
$$

so the new matrix of the form is $P^\top A P$. Since $A$ is symmetric, the
Spectral Theorem supplies an orthogonal $P$ with $P^\top A P = D$ diagonal. In
those coordinates the form becomes $y^\top D y$, a pure sum of squares with no
cross-terms.

> **Theorem (Principal Axes Theorem).** Let $A$ be an $n \times n$ symmetric
> matrix. There is an orthogonal change of variable $x = Py$ that transforms
> $x^\top A x$ into $y^\top D y = \lambda_1 y_1^2 + \cdots + \lambda_n y_n^2$,
> a quadratic form with no cross-product term. The columns of $P$ are the
> **principal axes** of the form.

> **Worked example.** Make the change of variable that removes the cross-term
> from $Q(x) = x_1^2 - 8x_1 x_2 - 5x_2^2$, whose matrix is
> $A = \begin{bmatrix} 1 & -4 \\ -4 & -5 \end{bmatrix}$. The eigenvalues are
> $\lambda = 3$ and $\lambda = -7$, with unit eigenvectors
> $$
> \lambda = 3: \; u_1 = \frac{1}{\sqrt5}\begin{bmatrix} 2 \\ -1 \end{bmatrix},
> \qquad
> \lambda = -7: \; u_2 = \frac{1}{\sqrt5}\begin{bmatrix} 1 \\ 2 \end{bmatrix}.
> $$
> Set $P = [\,u_1 \; u_2\,]$ and $D = \operatorname{diag}(3, -7)$. The change of
> variable $x = Py$ turns the form into
> $$
> x_1^2 - 8x_1 x_2 - 5x_2^2 = y^\top D y = 3y_1^2 - 7y_2^2.
> $$
> Both expressions take the same value at corresponding points. At $x = (2,-2)$,
> directly $Q = 4 - 8(2)(-2) - 5(4) = 4 + 32 - 20 = 16$; through the new
> variables $y = P^\top x = \tfrac{1}{\sqrt5}(6, -2)$, and
> $3(36/5) - 7(4/5) = (108 - 28)/5 = 16$.

## Principal axes are eigenvector axes

The geometry behind the algebra is a rotation. For an invertible $2 \times 2$
symmetric $A$ and a constant $c$, the set $\{x : x^\top A x = c\}$ is a conic:
an ellipse, a hyperbola, a pair of lines, a point, or empty. When $A$ is
diagonal the conic sits in **standard position**, its axes along the coordinate
directions. When $A$ has cross-terms, the same conic is rotated out of standard
position, and the principal axes — the eigenvectors of $A$ — mark the new
coordinate directions in which it straightens out.

$$
% caption: The equation xᵀAx = c is an ellipse rotated out of standard
% position; its principal axes are the eigenvectors u₁, u₂ of A, and in those
% coordinates it becomes the axis-aligned diagonal form λ₁y₁² + λ₂y₂² = c.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  % standard axes
  \draw[black] (-2.6,0) -- (2.6,0) node[right, black] {$x_1$};
  \draw[black] (0,-2.4) -- (0,2.4) node[above, black] {$x_2$};
  % rotated ellipse (major axis at ~ -35 deg)
  \draw[thick, rotate=-32] (0,0) ellipse (2.3 and 1.1);
  % principal axes (eigenvector directions)
  \draw[->, acc, very thick, rotate=-32] (0,0) -- (2.3,0) node[right, black] {$u_1$};
  \draw[->, acc, very thick, rotate=-32] (0,0) -- (0,1.1) node[above, black] {$u_2$};
  \node[black, rotate=-32] at (1.5,-0.35) {$y_1$};
  \node[black, rotate=-32] at (-0.4,0.75) {$y_2$};
  \node[black] at (1.9,1.9) {$x^\top A x = c$};
\end{tikzpicture}
$$

Finding the principal axes means finding coordinates in which the conic is
standard, the same decoupling that
[change of basis](/linear-algebra/vector-spaces/change-of-basis) performs
elsewhere.

$$
% caption: The change of variable y = Pᵀx rotates the tilted conic xᵀAx = c into
% the axis-aligned yᵀDy = c, whose axes are the coordinate directions e₁, e₂.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \begin{scope}[xshift=0cm]
    \draw[black] (-1.9,0) -- (1.9,0) node[right, black] {$x_1$};
    \draw[black] (0,-1.9) -- (0,1.9) node[above, black] {$x_2$};
    \draw[thick, rotate=32] (0,0) ellipse (1.6 and 0.85);
    \draw[->, acc, very thick, rotate=32] (0,0) -- (1.6,0) node[right, black] {$u_1$};
    \draw[->, acc, very thick, rotate=32] (0,0) -- (0,0.85) node[above left, black] {$u_2$};
    \node[black] at (0,-2.4) {tilted};
  \end{scope}
  \draw[->, black, thick] (2.5,0) -- (3.9,0) node[midway, above, black] {$y = P^T x$};
  \begin{scope}[xshift=6.4cm]
    \draw[black] (-1.9,0) -- (1.9,0) node[right, black] {$y_1$};
    \draw[black] (0,-1.9) -- (0,1.9) node[above, black] {$y_2$};
    \draw[thick] (0,0) ellipse (1.6 and 0.85);
    \draw[->, acc, very thick] (0,0) -- (1.6,0) node[below right, black] {$e_1$};
    \draw[->, acc, very thick] (0,0) -- (0,0.85) node[above, black] {$e_2$};
    \node[black] at (0,-2.4) {standard};
  \end{scope}
\end{tikzpicture}
$$

## Classifying quadratic forms

Once decoupled to $\lambda_1 y_1^2 + \cdots + \lambda_n y_n^2$, the sign of a
form is decided by the signs of the eigenvalues. The graph $z = Q(x)$ over
$\mathbb{R}^2$ makes the classification visible: a bowl opening upward when all
eigenvalues are positive, a bowl opening downward when all are negative, and a
saddle when the signs are mixed.

$$
% caption: The graph z = Q(x) is an upward bowl when Q is positive definite
% (values positive off the origin) and a saddle when Q is indefinite (positive
% along one principal axis, negative along the other).
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  % --- positive definite bowl (left) ---
  \begin{scope}[xshift=0cm]
    \draw[black, ->] (-1.5,0) -- (1.5,0);
    \draw[black, ->] (0,-0.3) -- (0,2.3) node[above, black] {$z$};
    \draw[acc, thick] plot[domain=-1.3:1.3, samples=40] (\x, {1.1*\x*\x});
    \node[black, align=center] at (0,-0.9) {positive def\/inite\\$z>0$ of\/f origin};
  \end{scope}
  % --- saddle (right) ---
  \begin{scope}[xshift=5cm]
    \draw[black, ->] (-1.6,0) -- (1.6,0);
    \draw[black, ->] (0,-1.4) -- (0,1.6) node[above, black] {$z$};
    \draw[thick] plot[domain=-1.2:1.2, samples=40] (\x, {1.0*\x*\x});
    \draw[thick, dashed] plot[domain=-1.2:1.2, samples=40] (\x, {-1.0*\x*\x});
    \node[black] at (1.35,1.25) {up axis};
    \node[black] at (1.35,-1.25) {down axis};
    \node[black, align=center] at (0,-2.0) {indef\/inite\\both signs};
  \end{scope}
\end{tikzpicture}
$$

> **Definition (Definiteness).** A quadratic form $Q$ is
>
> - **positive definite** if $Q(x) > 0$ for all $x \neq 0$;
> - **negative definite** if $Q(x) < 0$ for all $x \neq 0$;
> - **indefinite** if $Q(x)$ takes both positive and negative values;
> - **positive semidefinite** if $Q(x) \geq 0$ for all $x$, and **negative
>   semidefinite** if $Q(x) \leq 0$ for all $x$.

The classification carries over to the matrix: a positive definite matrix is a
symmetric matrix whose form is positive definite, and likewise for the other
terms. The eigenvalue criterion is immediate from the Principal Axes Theorem.

> **Theorem (Definiteness by eigenvalue sign).** For a symmetric matrix $A$,
> the form $x^\top A x$ is
>
> - positive definite iff every eigenvalue of $A$ is positive;
> - negative definite iff every eigenvalue is negative;
> - indefinite iff $A$ has both a positive and a negative eigenvalue.

> **Proof.** By the principal axes theorem there is an orthogonal $x = Py$ with
> $Q(x) = \lambda_1 y_1^2 + \cdots + \lambda_n y_n^2$. Because $P$ is invertible,
> $x$ ranges over all nonzero vectors exactly as $y$ does, so the values of $Q$
> are the values of the decoupled sum. That sum's sign is fixed by the signs of
> the $\lambda_i$: all positive forces $Q > 0$ off the origin, all
> negative forces $Q < 0$, and a mix of signs is realized by $y = e_i$ for the
> respective eigenvalues. $\;\blacksquare$

The semidefinite case is the boundary: $Q$ is positive semidefinite exactly when
every eigenvalue is nonnegative (allowing zeros), since a zero eigenvalue leaves
$Q = 0$ along its eigenvector while never turning negative.

The eigenvalue test corrects a naive reading of the coefficients.

> **Worked example.** The form
> $Q(x) = 3x_1^2 + 2x_2^2 + x_3^2 + 4x_1 x_2 + 4x_2 x_3$ has every visible
> coefficient positive, yet it is not positive definite. Its matrix
> $$
> A = \begin{bmatrix} 3 & 2 & 0 \\ 2 & 2 & 2 \\ 0 & 2 & 1 \end{bmatrix}
> $$
> has eigenvalues $5, 2, -1$. The negative eigenvalue makes $Q$ **indefinite**:
> the cross-terms overwhelm the diagonal along one direction. Only the
> eigenvalues settle the question.

$$
% caption: Seen from above, level curves xᵀAx = c are nested ellipses when Q is
% positive definite (same-sign eigenvalues) and crossing hyperbolas when Q is
% indefinite (opposite-sign eigenvalues).
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \begin{scope}[xshift=0cm]
    \draw[black] (-1.9,0) -- (1.9,0);
    \draw[black] (0,-1.9) -- (0,1.9);
    \draw[acc, thick] (0,0) ellipse (1.6 and 0.9);
    \draw[thin] (0,0) ellipse (1.05 and 0.59);
    \draw[thin] (0,0) ellipse (0.55 and 0.31);
    \node[black] at (0,-2.4) {positive def\/inite};
  \end{scope}
  \begin{scope}[xshift=5.6cm]
    \draw[black] (-1.9,0) -- (1.9,0);
    \draw[black] (0,-1.9) -- (0,1.9);
    \draw[acc, thick] plot[domain=0.45:1.75, samples=30] (\x, {0.65/\x});
    \draw[acc, thick] plot[domain=0.45:1.75, samples=30] (-\x, {-0.65/\x});
    \draw[thin] plot[domain=0.45:1.75, samples=30] (\x, {-0.65/\x});
    \draw[thin] plot[domain=0.45:1.75, samples=30] (-\x, {0.65/\x});
    \node[black] at (0,-2.4) {indef\/inite};
  \end{scope}
\end{tikzpicture}
$$

$$
% caption: The signs of the eigenvalues, not the signs of the coefficients,
% sort a form into its class.
\begin{tikzpicture}[scale=1.0, >=stealth, font=\footnotesize,
  box/.style={draw, minimum width=30mm, minimum height=9mm, align=center}]
  \definecolor{acc}{HTML}{4A6FA5}
  \node[box, draw=acc, text=acc, thick] (r) at (0,0) {signs of eigenvalues};
  \node[box] (a) at (-4.3,-1.9) {all positive};
  \node[box] (b) at (-1.45,-1.9) {all negative};
  \node[box] (c) at (1.45,-1.9) {none negative};
  \node[box] (d) at (4.3,-1.9) {mixed};
  \node[black] at (-4.3,-2.7) {positive def\/inite};
  \node[black] at (-1.45,-2.7) {negative def\/inite};
  \node[black] at (1.45,-2.7) {semidef\/inite};
  \node[black] at (4.3,-2.7) {indef\/inite};
  \draw[->, thick] (r) -- (a);
  \draw[->, thick] (r) -- (b);
  \draw[->, thick] (r) -- (c);
  \draw[->, thick] (r) -- (d);
\end{tikzpicture}
$$

## The 2×2 shortcut

For a $2 \times 2$ symmetric matrix $A = \begin{bmatrix} a & b \\ b & d
\end{bmatrix}$ with $\det A \neq 0$, the class can be read from the
determinant and the corner entry without computing eigenvalues. Writing the
characteristic polynomial two ways, $\det(A - \lambda I) = (\lambda -
\lambda_1)(\lambda - \lambda_2)$, and matching coefficients,

$$
\lambda_1 + \lambda_2 = a + d = \operatorname{tr} A,
\qquad
\lambda_1 \lambda_2 = ad - b^2 = \det A.
$$

The product of the eigenvalues is $\det A$ and their sum is the trace. Two same-
sign eigenvalues give a positive product; opposite signs give a negative one.

| Condition | Sign of eigenvalues | Class |
| --- | --- | --- |
| $\det A > 0$, $a > 0$ | both positive | positive definite |
| $\det A > 0$, $a < 0$ | both negative | negative definite |
| $\det A < 0$ | one of each | indefinite |

When $\det A > 0$ the eigenvalues share a sign, and the sign of the corner entry
$a = u_1^\top A u_1$ (with $u_1 = e_1$) reveals which. When $\det A < 0$ the
eigenvalues have opposite signs, so the form is indefinite regardless of the
diagonal.

## Positive definiteness and factorization

Positive definite matrices are the ones that behave like positive numbers, and
they are precisely the matrices admitting a "square-root" factorization. If $B$ is
$m \times n$ then $B^\top B$ is always positive semidefinite, because
$x^\top (B^\top B) x = (Bx)^\top(Bx) = \|Bx\|^2 \geq 0$; and if $B$ is square and
invertible then $Bx \neq 0$ for $x \neq 0$, making $B^\top B$ positive definite.
Conversely, a symmetric positive definite $A = PDP^\top$ can be written
$A = B^\top B$ by taking $B = P C P^\top$ with $C = D^{1/2}$ the diagonal matrix
of square roots of the eigenvalues.

A fast test for positive definiteness attempts to factor $A = R^\top R$ with $R$
upper triangular and positive diagonal entries. This
[Cholesky factorization](/linear-algebra/numerical-linear-algebra/lu-and-cholesky)
succeeds if and only if $A$ is positive definite, and it costs about half a
general elimination.