---
title: Matrices, Linear Systems, and the Eigenvalue Toolkit
module: Systems of First-Order Linear Equations
moduleNumber: 6
lessonNumber: 1
order: 601
summary: >
  Any nth-order linear equation, and any coupled collection of them, rewrites as a
  single first-order system x' = P(t)x + g(t). The matrix and vector algebra behind
  that form, the eigenvalue problem det(A - λI) = 0 that drives every solution
  method, and the fundamental theory — superposition, the Wronskian, Abel's
  theorem — together establish that n independent solutions span all solutions.
topics: [Systems of First-Order Linear Equations]
sources:
  - book: Boyce
    ref: "Ch. 7 — Systems of First-Order Linear Equations; §7.1 Introduction; §7.2 Matrices"
  - book: Boyce
    ref: "§7.3 Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors; §7.4 Basic Theory"
  - book: Simmons
    ref: "Ch. 10 §54 General Remarks on Systems; §55 Linear Systems"
draft: false
---

A system of differential equations governs several unknown functions of one
variable at once, each equation tying together the functions and their
derivatives. The current and voltage in a network, the masses in a coupled
mechanical system, the competing species in an ecosystem — each is a vector of
quantities evolving together, and the mathematics is a system of two or more
differential equations.[^boyce-intro] Two structural facts make the whole
subject tractable. First, every
equation can be put in _first-order_ form, so we never need to study anything
higher. Second, once written that way, a linear system is `x' = P(t)x + g(t)` —
a single vector equation whose theory mirrors, line for line, the single-equation
theory of the [second-order case](/differential-equations/second-order-linear/homogeneous-constant-coefficients).

## Reducing an equation to a first-order system

The reduction rests on one observation: an nth-order equation determines the
top derivative $y^{(n)}$ from the lower ones, so if we _name_ the lower
derivatives as new unknowns, each is the derivative of the one before it.

Take the general second-order linear equation, the spring-mass model
$m u'' + \gamma u' + k u = F(t)$. Set $x_1 = u$ and $x_2 = u'$. Then
$x_1' = u' = x_2$, and $x_2' = u''$ comes from solving the original equation:

$$
x_1' = x_2, \qquad
x_2' = -\frac{k}{m}\,x_1 - \frac{\gamma}{m}\,x_2 + \frac{1}{m}F(t).
$$

The one second-order equation has become two first-order equations. The same
scheme works at any order. For $y^{(n)} = F(t, y, y', \dots, y^{(n-1)})$,
introduce

$$
x_1 = y,\quad x_2 = y',\quad x_3 = y'',\quad \dots,\quad x_n = y^{(n-1)},
$$

so that $x_1' = x_2,\ x_2' = x_3,\ \dots,\ x_{n-1}' = x_n$, and the original
equation supplies the last one, $x_n' = F(t, x_1, \dots, x_n)$.

> **Worked example.** The spring-mass equation $u'' + \tfrac18 u' + u = 0$ becomes a
> first-order system by naming $x_1 = u$ and $x_2 = u'$.[^boyce-71ex] Then
> $x_1' = x_2$, and substituting $u = x_1$, $u' = x_2$, $u'' = x_2'$ into the
> equation gives $x_2' + \tfrac18 x_2 + x_1 = 0$, so
>
> $$
> x_1' = x_2, \qquad x_2' = -x_1 - \tfrac18 x_2.
> $$

> **Definition (First-order system).** A set of $n$ equations
> $x_i' = F_i(t, x_1, \dots, x_n)$, $i = 1, \dots, n$, for $n$ unknown functions
> of $t$. A **solution** on an interval $I$ is a set of differentiable functions
> $x_1 = \phi_1(t), \dots, x_n = \phi_n(t)$ satisfying every equation at every
> point of $I$. An **initial condition** prescribes the vector
> $\big(x_1(t_0), \dots, x_n(t_0)\big)$ at one point $t_0 \in I$.

$$
% caption: An nth-order equation becomes a first-order system by naming each
% lower derivative as a new unknown; the last equation carries the original
% dynamics.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  box/.style={draw, minimum width=34mm, minimum height=13mm, align=center},
  sub/.style={draw, minimum width=30mm, minimum height=13mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{4A6FA5}
  \node[box, draw=acc, text=acc, thick] (ode) at (0,0) {$\frac{d^2u}{dt^2} + p\frac{du}{dt} + qu = g$};
  \node[sub] (sub) at (0,-2.1) {let $x_1 = u,\ x_2 = \frac{du}{dt}$};
  \node[box] (s1) at (7.0,1.0) {$\frac{dx_1}{dt} = x_2$};
  \node[box] (s2) at (7.0,-1.0) {$\frac{dx_2}{dt} + p\,x_2 + q\,x_1 = g$};
  \draw[->, acc, thick] (ode) -- (sub);
  \draw[->, acc, thick] (sub.east) -- (s1.west);
  \draw[->, acc, thick] (sub.east) -- (s2.west);
  \node[font=\scriptsize, text=black, anchor=west] at (3.6,-2.1) {one equation, order 2};
\end{tikzpicture}
$$

The reduction has two payoffs. Almost every
[numerical integrator](/differential-equations/numerical/euler-and-runge-kutta)
is written for first-order systems, so the reduction is a prerequisite for
computing. And the vector viewpoint unifies equations of every order under a
single theory.

## Matrix and vector notation

Collecting $x_1, \dots, x_n$ into a column vector $\mathbf{x}$ and the
coefficients into a matrix turns the linear system into a single equation. The
most general system of $n$ first-order linear equations is

$$
x_i' = p_{i1}(t)x_1 + \cdots + p_{in}(t)x_n + g_i(t), \qquad i = 1, \dots, n,
$$

which in matrix form is compact:

$$
\mathbf{x}' = P(t)\,\mathbf{x} + \mathbf{g}(t),
\qquad
\mathbf{x} = \begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix},
\quad
P(t) = \big(p_{ij}(t)\big),
\quad
\mathbf{g}(t) = \begin{pmatrix} g_1 \\ \vdots \\ g_n \end{pmatrix}.
$$

The terminology follows the single-equation case exactly. The system is
**homogeneous** when $\mathbf{g}(t) \equiv \mathbf{0}$ and **nonhomogeneous**
otherwise. A matrix or vector function is continuous, differentiable, or
integrable when each entry is, and $\d A/\d t = (\d a_{ij}/\d t)$ entrywise. The
product rule still holds, but order matters, since matrices do not commute:

$$
\frac{\d}{\d t}(AB) = A\frac{\d B}{\d t} + \frac{\d A}{\d t}B.
$$

Two matrix facts underlie what follows, and both hinge on the
determinant. A square matrix $A$ is **nonsingular** (invertible) exactly when
$\det A \neq 0$; then $A\mathbf{x} = \mathbf{b}$ has the unique solution
$\mathbf{x} = A^{-1}\mathbf{b}$ for every $\mathbf{b}$. When $\det A = 0$ the
matrix is **singular**, and the homogeneous problem $A\mathbf{x} = \mathbf{0}$
has nonzero solutions — infinitely many of them. This dichotomy underlies the
eigenvalue problem, where an eigenvector is nothing other than a nonzero
solution of a homogeneous system of just this kind.

## Linear independence

Whether a collection of solutions is "enough" to build every solution is a
question of linear independence, defined first for vectors.

> **Definition (Linear independence).** Vectors
> $\mathbf{x}^{(1)}, \dots, \mathbf{x}^{(k)}$ are **linearly dependent** if there
> exist scalars $c_1, \dots, c_k$, not all zero, with
> $c_1 \mathbf{x}^{(1)} + \cdots + c_k \mathbf{x}^{(k)} = \mathbf{0}$. If the only
> such scalars are $c_1 = \cdots = c_k = 0$, the vectors are **linearly
> independent**.

For $n$ vectors each with $n$ components, assemble them as the columns of a
matrix $X$. The relation $c_1 \mathbf{x}^{(1)} + \cdots + c_n \mathbf{x}^{(n)} =
\mathbf{0}$ is then $X\mathbf{c} = \mathbf{0}$, which has only the trivial
solution $\mathbf{c} = \mathbf{0}$ precisely when $\det X \neq 0$. The test is
a single determinant:

$$
\mathbf{x}^{(1)}, \dots, \mathbf{x}^{(n)} \text{ linearly independent}
\iff \det X \neq 0.
$$

## Eigenvalues and eigenvectors

The equation $A\mathbf{x} = \mathbf{y}$ transforms a vector $\mathbf{x}$ into a
new vector $\mathbf{y}$. Most inputs come out pointing in a new direction. A few
special inputs keep their direction: for them $A\mathbf{x}$ is a scalar multiple
of $\mathbf{x}$. These directions are what solve constant-coefficient systems,
since integration becomes available precisely when $A\mathbf{x}$ reduces to a scalar multiple of $\mathbf{x}$.

> **Definition (Eigenvalue, eigenvector).** A scalar $\lambda$ is an
> **eigenvalue** of the square matrix $A$ if
> $A\boldsymbol{\xi} = \lambda\boldsymbol{\xi}$ for some nonzero vector
> $\boldsymbol{\xi}$, which is then an **eigenvector** belonging to $\lambda$.
> Equivalently $(A - \lambda I)\boldsymbol{\xi} = \mathbf{0}$ with
> $\boldsymbol{\xi} \neq \mathbf{0}$.

Since $(A - \lambda I)\boldsymbol{\xi} = \mathbf{0}$ must have a nonzero
solution, the matrix $A - \lambda I$ must be singular. That is the whole
computation:

> **Definition (Characteristic equation).** The eigenvalues of an $n \times n$
> matrix $A$ are the roots of the degree-$n$ polynomial equation
> $$
> \det(A - \lambda I) = 0.
> $$
> Each eigenvector belonging to $\lambda$ is a nonzero solution of
> $(A - \lambda I)\boldsymbol{\xi} = \mathbf{0}$, determined only up to a nonzero
> scalar multiple.

$$
% caption: Most vectors are rotated by A (left); an eigenvector keeps its
% direction, and A only rescales it (right) by the eigenvalue.
\begin{tikzpicture}[>=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \definecolor{hot}{HTML}{C0603A}
  % left panel: generic vector rotated
  \begin{scope}
    \draw[black, ->] (-0.3,0) -- (3.3,0) node[right, black]{$x_1$};
    \draw[black, ->] (0,-0.3) -- (0,3.3) node[above, black]{$x_2$};
    \draw[acc, very thick, ->] (0,0) -- (2.4,0.7) node[right]{$\mathbf{x}$};
    \draw[hot, very thick, ->] (0,0) -- (1.0,2.5) node[above]{$A\mathbf{x}$};
    \node[font=\scriptsize, text=black] at (1.6,-0.8) {direction changes};
  \end{scope}
  % right panel: eigenvector rescaled
  \begin{scope}[xshift=6cm]
    \draw[black, ->] (-0.3,0) -- (3.3,0) node[right, black]{$x_1$};
    \draw[black, ->] (0,-0.3) -- (0,3.3) node[above, black]{$x_2$};
    \draw[acc, very thick, ->] (0,0) -- (1.1,1.1) node[above left]{$\mathbf{v}$};
    \draw[hot, very thick, ->] (0,0) -- (2.6,2.6) node[above right]{$A\mathbf{v}$};
    \node[font=\scriptsize, text=black] at (1.6,-0.8) {same line, rescaled};
  \end{scope}
\end{tikzpicture}
$$

> **Worked example.** For $A = \begin{pmatrix} 3 & -1 \\ 4 & -2 \end{pmatrix}$, the
> characteristic equation is[^boyce-73ex4]
>
> $$
> \det(A - \lambda I) =
> \begin{vmatrix} 3 - \lambda & -1 \\ 4 & -2 - \lambda \end{vmatrix}
> = \lambda^2 - \lambda - 2 = (\lambda - 2)(\lambda + 1) = 0,
> $$
>
> so $\lambda_1 = 2$ and $\lambda_2 = -1$. For $\lambda_1 = 2$ the equation
> $(A - 2I)\boldsymbol{\xi} = \mathbf{0}$ reduces to the single condition
> $\xi_1 - \xi_2 = 0$, giving $\boldsymbol{\xi}^{(1)} = (1, 1)^\top$. For
> $\lambda_2 = -1$ the condition is $4\xi_1 - \xi_2 = 0$, giving
> $\boldsymbol{\xi}^{(2)} = (1, 4)^\top$. Any nonzero multiple of either vector is
> equally an eigenvector; small integer representatives are the conventional choice.

### Multiplicity

A degree-$n$ characteristic polynomial has $n$ roots counted with repetition,
so an $n \times n$ matrix has $n$ eigenvalues, some possibly equal. When roots
coincide, two different counts diverge, and the gap between them determines
whether the eigenvalue is defective.

- **Algebraic multiplicity** $m$: the number of times $\lambda$ appears as a
  root of $\det(A - \lambda I) = 0$.
- **Geometric multiplicity** $q$: the number of linearly independent
  eigenvectors belonging to $\lambda$.

$$
% caption: Algebraic multiplicity m counts a root's repetitions; geometric
% multiplicity q counts its independent eigenvectors, always between 1 and m. When
% q falls short of m the eigenvalue is defective, missing m minus q eigenvectors.
\begin{tikzpicture}[>=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{4A6FA5}
  \definecolor{hot}{HTML}{C0603A}
  \node[anchor=east, text=black] at (-0.3,2.0) {algebraic $m = 3$};
  \foreach \i in {0,1,2} {
    \draw[acc, thick, fill=acc!12] (\i*1.2,1.6) rectangle (\i*1.2+1.0,2.4);
    \node[acc] at (\i*1.2+0.5,2.0) {$r$};
  }
  \node[anchor=east, text=black] at (-0.3,0.4) {geometric $q = 2$};
  \draw[acc, thick, fill=acc!12] (0,0) rectangle (1.0,0.8);
  \node[acc] at (0.5,0.4) {$\mathbf{v}_1$};
  \draw[acc, thick, fill=acc!12] (1.2,0) rectangle (2.2,0.8);
  \node[acc] at (1.7,0.4) {$\mathbf{v}_2$};
  \draw[hot, thick, dashed] (2.4,0) rectangle (3.4,0.8);
  \node[hot] at (2.9,0.4) {?};
  \node[hot, anchor=west, font=\scriptsize] at (3.6,0.4) {missing eigenvector};
\end{tikzpicture}
$$

These always satisfy $1 \le q \le m$: an eigenvalue has at least one eigenvector
and never more independent ones than its algebraic multiplicity. When $q < m$
the eigenvalue is **defective**, short of eigenvectors, and the
[repeated-eigenvalue methods](/differential-equations/systems/repeated-eigenvalues-fundamental-matrices)
of a later lesson are needed. Two facts keep the common cases simple:

- **Distinct eigenvalues give independent eigenvectors.** If
  $\lambda_1, \dots, \lambda_k$ are distinct, their eigenvectors
  $\boldsymbol{\xi}^{(1)}, \dots, \boldsymbol{\xi}^{(k)}$ are linearly
  independent. So if all $n$ eigenvalues are simple, $A$ has a full set of $n$
  independent eigenvectors.
- **Symmetric matrices never go defective.** If $A$ is real symmetric
  ($A^\top = A$), all eigenvalues are real and a full set of $n$ independent
  (in fact orthogonal) eigenvectors always exists, even with repeated
  eigenvalues.

| Case | Algebraic $m$ | Geometric $q$ | Independent eigenvectors |
| --- | --- | --- | --- |
| All eigenvalues distinct | each $1$ | each $1$ | full set of $n$ |
| Repeated, non-defective | $m > 1$ | $q = m$ | full set of $n$ |
| Repeated, defective | $m > 1$ | $q < m$ | fewer than $n$ |
| Real symmetric $A$ | any | $q = m$ always | full set of $n$, orthogonal |

> **Worked example.** The real symmetric matrix
> $A = \begin{pmatrix} 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{pmatrix}$ has
> characteristic equation[^boyce-73ex5]
>
> $$
> \det(A - \lambda I) = -\lambda^3 + 3\lambda + 2 = -(\lambda - 2)(\lambda + 1)^2 = 0,
> $$
>
> so $\lambda_1 = 2$ is simple and $\lambda = -1$ is a double root. For
> $\lambda = -1$, the system $(A + I)\boldsymbol{\xi} = \mathbf{0}$ reduces to the
> single condition $\xi_1 + \xi_2 + \xi_3 = 0$, a plane of solutions spanned by two
> independent eigenvectors, for instance $(1, 0, -1)^\top$ and $(0, 1, -1)^\top$.
> For $\lambda_1 = 2$, $(A - 2I)\boldsymbol{\xi} = \mathbf{0}$ gives
> $\boldsymbol{\xi} = (1, 1, 1)^\top$. The double eigenvalue furnishes its full two
> eigenvectors, so the complete set of three is available: a real symmetric matrix
> is never defective.

## The fundamental theory of homogeneous systems

Consider the homogeneous linear system:

$$
\mathbf{x}' = P(t)\,\mathbf{x}.
$$

Its solution theory is a vector translation of the second-order theory, and it
rests on the existence-uniqueness guarantee for linear systems: if $P$ and
$\mathbf{g}$ are continuous on an open interval $I$, then for any $t_0 \in I$
and any prescribed initial vector there is a unique solution, and it exists
throughout $I$.[^boyce-euthm] Unlike the nonlinear case, no solution can blow up
inside the interval of continuity.

Everything then follows from one closure property.

> **Theorem (Principle of superposition).** If $\mathbf{x}^{(1)}$ and
> $\mathbf{x}^{(2)}$ solve $\mathbf{x}' = P(t)\mathbf{x}$, then every linear
> combination $c_1\mathbf{x}^{(1)} + c_2\mathbf{x}^{(2)}$ solves it too, for any
> constants $c_1, c_2$. By repetition, any finite combination of solutions is a
> solution.

Superposition says the solutions form a vector space. The question is its
dimension — how many independent solutions must be combined to reach all of
them. The determinant test for independence, applied to solution vectors, gets
its own name.

> **Definition (Wronskian).** For $n$ solutions
> $\mathbf{x}^{(1)}, \dots, \mathbf{x}^{(n)}$ of $\mathbf{x}' = P(t)\mathbf{x}$,
> arranged as the columns of a matrix $X(t)$, the **Wronskian** is
> $$
> W[\mathbf{x}^{(1)}, \dots, \mathbf{x}^{(n)}](t) = \det X(t).
> $$
> The solutions are linearly independent at a point exactly when $W \neq 0$
> there.

A set of $n$ solutions that is independent throughout $I$ is a **fundamental
set**, and it is a basis: every solution is one of its combinations, uniquely.

> **Theorem (General solution).** If $\mathbf{x}^{(1)}, \dots, \mathbf{x}^{(n)}$
> are linearly independent solutions of $\mathbf{x}' = P(t)\mathbf{x}$ on $I$,
> then every solution can be written
> $$
> \mathbf{x}(t) = c_1\mathbf{x}^{(1)}(t) + \cdots + c_n\mathbf{x}^{(n)}(t)
> $$
> in exactly one way. This expression, with $c_1, \dots, c_n$ arbitrary, is the
> **general solution**.

The proof invokes the existence-uniqueness theorem: at a point $t_0$,
choosing the $c_i$ to match a prescribed $\mathbf{x}(t_0)$ is a linear system
whose coefficient determinant is $W(t_0) \neq 0$, so the $c_i$ exist and are
unique; the resulting combination and the given solution then agree at $t_0$ and
so coincide everywhere.

$$
% caption: n independent solution vectors are the columns of X(t); their
% Wronskian det X(t) being nonzero certifies them as a basis for every solution.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  col/.style={draw, minimum width=9mm, minimum height=20mm, align=center}]
  \definecolor{acc}{HTML}{4A6FA5}
  \node[col, fill=acc!10] (c1) at (0,0) {$\mathbf{x}^{(1)}$};
  \node[col, fill=acc!10] (c2) at (1.0,0) {$\mathbf{x}^{(2)}$};
  \node[font=\small] (dots) at (1.9,0) {...};
  \node[col, fill=acc!10] (cn) at (2.8,0) {$\mathbf{x}^{(n)}$};
  \node[font=\scriptsize, text=black, anchor=north] at (1.4,-1.25) {columns = solutions};
  \node[left=3mm of c1, font=\small] {$X(t) =$};
  \draw[->, acc, thick] (3.9,0) -- (5.3,0);
  \node[draw, draw=acc, thick, minimum width=26mm, minimum height=13mm, align=center, font=\scriptsize] (w) at (7.0,0) {$\det X(t)$ nonzero\\on $I$};
  \draw[->, acc, thick] (8.5,0) -- (9.9,0);
  \node[align=center, font=\scriptsize] at (11.4,0) {fundamental\\set (basis)};
\end{tikzpicture}
$$

One more theorem removes the burden of checking the Wronskian everywhere.

> **Theorem (Abel's theorem).** If $\mathbf{x}^{(1)}, \dots, \mathbf{x}^{(n)}$
> solve $\mathbf{x}' = P(t)\mathbf{x}$ on $I$, their Wronskian satisfies
> $$
> W(t) = c\,\exp\!\left(\int \big[p_{11}(t) + \cdots + p_{nn}(t)\big]\,\d t\right),
> $$
> so $W$ is either identically zero or never zero on $I$.

Because the exponential is never zero, the constant $c$ alone determines $W$:
it vanishes nowhere or everywhere. A fundamental set can therefore be certified
by evaluating $W$ at a single convenient point. The trace $p_{11} + \cdots +
p_{nn}$ appearing in the exponent is the same quantity that will govern
stability once the coefficients are constant.

## Summary

Three moves recur throughout the module.

- **Reduce.** Any linear equation or coupled set becomes a first-order system
  $\mathbf{x}' = P(t)\mathbf{x} + \mathbf{g}(t)$ by naming derivatives as new
  unknowns.
- **Diagonalize the dynamics.** For constant $A$, the eigenvalue problem
  $\det(A - \lambda I) = 0$ finds the special directions $\boldsymbol{\xi}$ in
  which the system decouples into scalar exponential behavior.
- **Assemble.** $n$ independent solutions, certified by a nonzero Wronskian
  (Abel: check one point), form a basis whose combinations are the general
  solution.

[^boyce-intro]: **Boyce**, _Elementary Differential Equations and Boundary Value Problems_, §7.1 — Introduction: systems arising from mechanical and electrical networks, and the reduction of an $n$th-order equation to $n$ first-order equations. See also Simmons, §54, on the general setting.
[^boyce-euthm]: **Boyce**, §7.1, Theorem 7.1.2 (existence and uniqueness for linear systems): continuity of $P$ and $\mathbf{g}$ on $I$ guarantees a unique solution through any point, existing on all of $I$; and §7.4 for the superposition, Wronskian, general-solution, and Abel theorems.
[^boyce-71ex]: **Boyce**, §7.1, Example 1: the spring-mass equation $u'' + \tfrac18 u' + u = 0$ rewritten as a first-order system.
[^boyce-73ex4]: **Boyce**, §7.3, Example 4: eigenvalues and eigenvectors of $\begin{pmatrix} 3 & -1 \\ 4 & -2 \end{pmatrix}$.
[^boyce-73ex5]: **Boyce**, §7.3, Example 5: eigenvalues and eigenvectors of the real symmetric matrix with characteristic equation $-\lambda^3 + 3\lambda + 2 = 0$, illustrating a non-defective repeated eigenvalue.