Matrices, Linear Systems, and the Eigenvalue Toolkit

A system of differential equations governs several unknown functions of one variable at once, each equation tying together the functions and their derivatives. The current and voltage in a network, the masses in a coupled mechanical system, the competing species in an ecosystem — each is a vector of quantities evolving together, and the mathematics is a system of two or more differential equations.¹ Two structural facts make the whole subject tractable. First, every equation can be put in first-order form, so we never need to study anything higher. Second, once written that way, a linear system is x' = P(t)x + g(t) — a single vector equation whose theory mirrors, line for line, the single-equation theory of the second-order case.

Reducing an equation to a first-order system

The reduction rests on one observation: an nth-order equation determines the top derivative $y^{(n)}$ from the lower ones, so if we name the lower derivatives as new unknowns, each is the derivative of the one before it.

Take the general second-order linear equation, the spring-mass model $m{u}^{\prime \prime }+\gamma u^{\prime }+ku=F(t)$. Set $x_1=u$ and $x_2=u^{\prime }$. Then $x_1^{\prime }=u^{\prime }=x_2$, and $x_2^{\prime }=u^{\prime \prime }$ comes from solving the original equation:

The one second-order equation has become two first-order equations. The same scheme works at any order. For $y^{(n)}=F(t,y,y^{\prime },\dots ,y^{(n-1)})$, introduce

so that $x_1^{\prime }=x_2,x_2^{\prime }=x_3,\dots ,x_{n-1}^{\prime }=x_n$, and the original equation supplies the last one, $x_n^{\prime }=F(t,x_1,\dots ,x_n)$.

The reduction has two payoffs. Almost every numerical integrator is written for first-order systems, so the reduction is a prerequisite for computing. And the vector viewpoint unifies equations of every order under a single theory.

Matrix and vector notation

Collecting $x_1,\dots ,x_n$ into a column vector $\mathbf{x}$ and the coefficients into a matrix turns the linear system into a single equation. The most general system of $n$ first-order linear equations is

which in matrix form is compact:

The terminology follows the single-equation case exactly. The system is homogeneous when $\mathbf{g}(t)\equiv 0$ and nonhomogeneous otherwise. A matrix or vector function is continuous, differentiable, or integrable when each entry is, and $\mathbf{d}A/\mathbf{d}t=(\mathbf{d}{a}_{ij}/\mathbf{d}t)$ entrywise. The product rule still holds, but order matters, since matrices do not commute:

Two matrix facts underlie what follows, and both hinge on the determinant. A square matrix $A$ is nonsingular (invertible) exactly when $\det A\ne 0$; then $A\mathbf{x}=\mathbf{b}$ has the unique solution $\mathbf{x}=A^{-1}\mathbf{b}$ for every $\mathbf{b}$. When $\det A=0$ the matrix is singular, and the homogeneous problem $A\mathbf{x}=0$ has nonzero solutions — infinitely many of them. This dichotomy underlies the eigenvalue problem, where an eigenvector is nothing other than a nonzero solution of a homogeneous system of just this kind.

Linear independence

Whether a collection of solutions is enough to build every solution is a question of linear independence, defined first for vectors.

For $n$ vectors each with $n$ components, assemble them as the columns of a matrix $X$. The relation $c_1{\mathbf{x}}^{(1)}+\cdots +c_n{\mathbf{x}}^{(n)}=0$ is then $X\mathbf{c}=0$, which has only the trivial solution $\mathbf{c}=0$ precisely when $\det X\ne 0$. The test is a single determinant:

Eigenvalues and eigenvectors

The equation $A\mathbf{x}=\mathbf{y}$ transforms a vector $\mathbf{x}$ into a new vector $\mathbf{y}$. Most inputs come out pointing in a new direction. A few special inputs keep their direction: for them $A\mathbf{x}$ is a scalar multiple of $\mathbf{x}$. These directions are what solve constant-coefficient systems, since integration becomes available precisely when $A\mathbf{x}$ reduces to a scalar multiple of $\mathbf{x}$.

Since $(A-\lambda I)\mathbf{\xi }=0$ must have a nonzero solution, the matrix $A-\lambda I$ must be singular. That is the whole computation:

Multiplicity

A degree-$n$ characteristic polynomial has $n$ roots counted with repetition, so an $n\times n$ matrix has $n$ eigenvalues, some possibly equal. When roots coincide, two different counts diverge, and the gap between them determines whether the eigenvalue is defective.

• Algebraic multiplicity $m$: the number of times $\lambda$ appears as a root of $\det (A-\lambda I)=0$.
• Geometric multiplicity $q$: the number of linearly independent eigenvectors belonging to $\lambda$.

These always satisfy $1\le q\le m$: an eigenvalue has at least one eigenvector and never more independent ones than its algebraic multiplicity. When $q<m$ the eigenvalue is defective, short of eigenvectors, and the repeated-eigenvalue methods of a later lesson are needed. Two facts keep the common cases simple:

• Distinct eigenvalues give independent eigenvectors. If ${\lambda }_1,\dots ,{\lambda }_k$ are distinct, their eigenvectors ${\mathbf{\xi }}^{(1)},\dots ,{\mathbf{\xi }}^{(k)}$ are linearly independent. So if all $n$ eigenvalues are simple, $A$ has a full set of $n$ independent eigenvectors.
• Symmetric matrices never go defective. If $A$ is real symmetric ($A^{\top }=A$), all eigenvalues are real and a full set of $n$ independent (in fact orthogonal) eigenvectors always exists, even with repeated eigenvalues.

The fundamental theory of homogeneous systems

Consider the homogeneous linear system:

Its solution theory is a vector translation of the second-order theory, and it rests on the existence-uniqueness guarantee for linear systems: if $P$ and $\mathbf{g}$ are continuous on an open interval $I$, then for any $t_0\in I$ and any prescribed initial vector there is a unique solution, and it exists throughout $I$.⁵ Unlike the nonlinear case, no solution can blow up inside the interval of continuity.

Everything then follows from one closure property.

Superposition says the solutions form a vector space. The question is its dimension — how many independent solutions must be combined to reach all of them. The determinant test for independence, applied to solution vectors, gets its own name.

A set of $n$ solutions that is independent throughout $I$ is a fundamental set, and it is a basis: every solution is one of its combinations, uniquely.

The proof invokes the existence-uniqueness theorem: at a point $t_0$, choosing the $c_i$ to match a prescribed $\mathbf{x}(t_0)$ is a linear system whose coefficient determinant is $W(t_0)\ne 0$, so the $c_i$ exist and are unique; the resulting combination and the given solution then agree at $t_0$ and so coincide everywhere.

One more theorem removes the burden of checking the Wronskian everywhere.

Because the exponential is never zero, the constant $c$ alone determines $W$: it vanishes nowhere or everywhere. A fundamental set can therefore be certified by evaluating $W$ at a single convenient point. The trace $p_{11}+\cdots +p_{nn}$ appearing in the exponent is the same quantity that will govern stability once the coefficients are constant.

Summary

Three moves recur throughout the module.

• Reduce. Any linear equation or coupled set becomes a first-order system ${\mathbf{x}}^{\prime }=P(t)\mathbf{x}+\mathbf{g}(t)$ by naming derivatives as new unknowns.
• Diagonalize the dynamics. For constant $A$, the eigenvalue problem $\det (A-\lambda I)=0$ finds the special directions $\mathbf{\xi }$ in which the system decouples into scalar exponential behavior.
• Assemble. $n$ independent solutions, certified by a nonzero Wronskian (Abel: check one point), form a basis whose combinations are the general solution.