Quadratic Forms

Linear systems involve expressions of degree one in the variables. The next step up is degree two. Sums of squares already appeared in the norm $∥ x ∥^{2} = x^{⊤} x$ ; a general second-degree homogeneous expression, allowing cross-products $x_{i} x_{j}$ , is a quadratic form. These occur throughout applications: as energies in physics, as the objective in optimization, as the noise power of a signal, and as the covariance ellipsoids of statistics. Symmetric-matrix theory supplies the tools to analyze them.

The simplest nonzero form is $Q (x) = x^{⊤} I x = ∥ x ∥^{2}$ . In general, $x^{⊤} A x$ is a scalar assembled from products of the coordinates, with the diagonal of $A$ supplying the squared terms and the off-diagonal entries the cross-terms.

From matrix to form and back

Multiplying out $x^{⊤} A x$ for a $2 \times 2$ matrix shows how the entries enter. With $x = (x_{1}, x_{2})$ ,

[x_{1} x_{2}] [3 - 2 - 2 7] [x_{1} x_{2}] = 3 x_{1}^{2} - 4 x_{1} x_{2} + 7 x_{2}^{2} .

The squared coefficients $3$ and $7$ come from the diagonal. The cross-term $- 4 x_{1} x_{2}$ collects both off-diagonal entries: the $- 2$ in position $(1, 2)$ and the matching $- 2$ in position $(2, 1)$ each contribute $- 2 x_{1} x_{2}$ . A diagonal matrix produces no cross-term at all.

Reversing the direction, any quadratic form has a unique symmetric matrix. The rule: the coefficient of $x_{i}^{2}$ goes on the diagonal, and the coefficient of $x_{i} x_{j}$ (for $i \neq = j$ ) is split evenly between positions $(i, j)$ and $(j, i)$ . For the three-variable form $Q (x) = 5 x_{1}^{2} + 3 x_{2}^{2} + 2 x_{3}^{2} - x_{1} x_{2} + 8 x_{2} x_{3}$ ,

A = 5 - 1/2 0 - 1/2 34 042 .

The $x_{1} x_{2}$ coefficient $- 1$ splits into two $- \frac{1}{2}$ entries; the $x_{2} x_{3}$ coefficient $8$ splits into two $4$ entries; the absent $x_{1} x_{3}$ term leaves zeros in positions $(1, 3)$ and $(3, 1)$ . Splitting evenly is what keeps $A$ symmetric, and symmetry is what makes the Spectral Theorem available.

Removing the cross-terms

Cross-terms make a quadratic form awkward to read: the sign of $Q$ and its extreme values are obscured by the coupling between variables. A change of variable removes them.

Substituting $x = P y$ into $x^{⊤} A x$ and collecting the matrix,

x^{⊤} A x = (P y)^{⊤} A (P y) = y^{⊤} (P^{⊤} A P) y,

so the new matrix of the form is $P^{⊤} A P$ . Since $A$ is symmetric, the Spectral Theorem supplies an orthogonal $P$ with $P^{⊤} A P = D$ diagonal. In those coordinates the form becomes $y^{⊤} D y$ , a pure sum of squares with no cross-terms.

Principal axes are eigenvector axes

The geometry behind the algebra is a rotation. For an invertible $2 \times 2$ symmetric $A$ and a constant $c$ , the set ${x : x^{⊤} A x = c}$ is a conic: an ellipse, a hyperbola, a pair of lines, a point, or empty. When $A$ is diagonal the conic sits in standard position, its axes along the coordinate directions. When $A$ has cross-terms, the same conic is rotated out of standard position, and the principal axes — the eigenvectors of $A$ — mark the new coordinate directions in which it straightens out.

The equation xᵀAx = c is an ellipse rotated out of standard position; its principal axes are the eigenvectors u₁, u₂ of A, and in those coordinates it becomes the axis-aligned diagonal form λ₁y₁² + λ₂y₂² = c.

Finding the principal axes means finding coordinates in which the conic is standard, the same decoupling that change of basis performs elsewhere.

The change of variable y = Pᵀx rotates the tilted conic xᵀAx = c into the axis-aligned yᵀDy = c, whose axes are the coordinate directions e₁, e₂.

Classifying quadratic forms

Once decoupled to $λ_{1} y_{1}^{2} + \dots + λ_{n} y_{n}^{2}$ , the sign of a form is decided by the signs of the eigenvalues. The graph $z = Q (x)$ over $R^{2}$ makes the classification visible: a bowl opening upward when all eigenvalues are positive, a bowl opening downward when all are negative, and a saddle when the signs are mixed.

The graph z = Q(x) is an upward bowl when Q is positive definite (values positive off the origin) and a saddle when Q is indefinite (positive along one principal axis, negative along the other).

The classification carries over to the matrix: a positive definite matrix is a symmetric matrix whose form is positive definite, and likewise for the other terms. The eigenvalue criterion is immediate from the Principal Axes Theorem.

The semidefinite case is the boundary: $Q$ is positive semidefinite exactly when every eigenvalue is nonnegative (allowing zeros), since a zero eigenvalue leaves $Q = 0$ along its eigenvector while never turning negative.

The eigenvalue test corrects a naive reading of the coefficients.

Seen from above, level curves xᵀAx = c are nested ellipses when Q is positive definite (same-sign eigenvalues) and crossing hyperbolas when Q is indefinite (opposite-sign eigenvalues).

The signs of the eigenvalues, not the signs of the coefficients, sort a form into its class.

The 2×2 shortcut

For a $2 \times 2$ symmetric matrix $A = [a b b d]$ with $det A \neq = 0$ , the class can be read from the determinant and the corner entry without computing eigenvalues. Writing the characteristic polynomial two ways, $det (A - λ I) = (λ - λ_{1}) (λ - λ_{2})$ , and matching coefficients,

λ_{1} + λ_{2} = a + d = tr A, λ_{1} λ_{2} = a d - b^{2} = det A .

The product of the eigenvalues is $det A$ and their sum is the trace. Two same- sign eigenvalues give a positive product; opposite signs give a negative one.

Condition	Sign of eigenvalues	Class
$det A > 0$ , $a > 0$	both positive	positive definite
$det A > 0$ , $a < 0$	both negative	negative definite
$det A < 0$	one of each	indefinite

When $det A > 0$ the eigenvalues share a sign, and the sign of the corner entry $a = u_{1}^{⊤} A u_{1}$ (with $u_{1} = e_{1}$ ) reveals which. When $det A < 0$ the eigenvalues have opposite signs, so the form is indefinite regardless of the diagonal.

Positive definiteness and factorization

Positive definite matrices are the ones that behave like positive numbers, and they are precisely the matrices admitting a square-root factorization. If $B$ is $m \times n$ then $B^{⊤} B$ is always positive semidefinite, because $x^{⊤} (B^{⊤} B) x = (B x)^{⊤} (B x) = ∥ B x ∥^{2} \geq 0$ ; and if $B$ is square and invertible then $B x \neq = 0$ for $x \neq = 0$ , making $B^{⊤} B$ positive definite. Conversely, a symmetric positive definite $A = P D P^{⊤}$ can be written $A = B^{⊤} B$ by taking $B = P C P^{⊤}$ with $C = D^{1/2}$ the diagonal matrix of square roots of the eigenvalues.

A fast test for positive definiteness attempts to factor $A = R^{⊤} R$ with $R$ upper triangular and positive diagonal entries. This Cholesky factorization succeeds if and only if $A$ is positive definite, and it costs about half a general elimination.