---
title: Vector Equations and the Matrix Equation Ax = b
module: Linear Equations in Linear Algebra
moduleNumber: 1
lessonNumber: 2
order: 102
summary: >
  The same linear system reads three equivalent ways: a system of equations, a
  vector equation asking whether b is a linear combination of fixed vectors, and
  a matrix equation Ax = b. Ax is the linear combination of A's columns weighted
  by x, so consistency for a given b means b lies in the span of the columns, and
  consistency for every b means the columns span all of R^m.
topics: [Linear Equations in Linear Algebra]
sources:
  - book: Lay
    ref: "Ch. 1 — Linear Equations; §1.3 Vector Equations"
  - book: Lay
    ref: "§1.4 The Matrix Equation Ax = b"
---

A linear system can be read through the columns of its coefficient matrix rather
than its rows. Two questions come with this view: whether a target vector is
reachable as a combination of given vectors, and whether a fixed set of vectors
reaches everything. Both restate the consistency question, and both are answered
by pivots.

## Vectors in $\mathbb{R}^n$

A matrix with a single column is a **column vector**, or simply a **vector**. For
now a vector is an ordered list of numbers written vertically; the general
definition comes with [vector
spaces](/linear-algebra/vector-spaces/vector-spaces-and-subspaces). The set of
all vectors with $n$ real entries is $\mathbb{R}^n$,

$$
\mathbf{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} \in \mathbb{R}^n.
$$

Two vectors are equal exactly when their corresponding entries agree, so order
matters: $(4, 7) \ne (7, 4)$. Two operations are defined entrywise.

> **Definition (Vector addition and scalar multiplication).** For $\mathbf{u},
> \mathbf{v} \in \mathbb{R}^n$ and a scalar $c$, the **sum** $\mathbf{u} +
> \mathbf{v}$ adds corresponding entries, and the **scalar multiple** $c\mathbf{u}$
> multiplies every entry of $\mathbf{u}$ by $c$. The **zero vector** $\mathbf{0}$
> has all entries zero.

A scalar is written in lightface to distinguish it from a boldface vector. In
$\mathbb{R}^2$ the operations have a picture: identify the vector $(a, b)$ with
the point $(a, b)$ in the plane, or with the arrow from the origin to that point.
Addition is then the **parallelogram rule**, and scalar multiplication stretches
or reverses the arrow.

$$
% caption: Addition of two vectors completes the parallelogram on 0, u, v; a
% scalar multiple c u stretches u by |c| and reverses it when c is negative.
\begin{tikzpicture}[scale=0.9, font=\footnotesize, >=stealth]
\definecolor{acc}{HTML}{4A6FA5}
% --- left: parallelogram rule ---
\begin{scope}
  \draw[black, ->] (-0.3,0) -- (4.2,0);
  \draw[black, ->] (0,-0.3) -- (0,3.2);
  \coordinate (u) at (3,1);
  \coordinate (v) at (1,2.4);
  \coordinate (s) at (4,3.4);
  \draw[thick, ->] (0,0) -- (u) node[below right] {$\mathbf{u}$};
  \draw[thick, ->] (0,0) -- (v) node[above left] {$\mathbf{v}$};
  \draw[acc, thick, ->] (0,0) -- (s) node[above right] {$\mathbf{u}+\mathbf{v}$};
  \draw[black, dashed] (u) -- (s);
  \draw[black, dashed] (v) -- (s);
\end{scope}
% --- right: scalar multiples ---
\begin{scope}[xshift=6cm]
  \draw[black, ->] (-1.8,0) -- (2.2,0);
  \draw[black, ->] (0,-1.2) -- (0,3.4);
  \coordinate (u2) at (1,1.4);
  \draw[acc, thick, ->] (0,0) -- (2,2.8) node[above right] {$2\mathbf{u}$};
  \draw[thick, ->] (0,0) -- (u2) node[right] {$\mathbf{u}$};
  \draw[acc, thick, ->] (0,0) -- (-0.66,-0.93) node[below left] {$-\tfrac23\mathbf{u}$};
\end{scope}
\end{tikzpicture}
$$

The eight familiar rules — commutativity, associativity, distributivity, and so
on — hold entry by entry because they hold for real numbers. They are the
axioms that later define a vector space; here they are theorems about lists.

## Linear combinations and span

Scaling and adding, combined, produce the central construction of the subject.

> **Definition (Linear combination).** Given vectors $\mathbf{v}_1, \dots,
> \mathbf{v}_p$ in $\mathbb{R}^n$ and scalars $c_1, \dots, c_p$ (the **weights**),
> the vector
> $$
> \mathbf{y} = c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_p \mathbf{v}_p
> $$
> is a **linear combination** of $\mathbf{v}_1, \dots, \mathbf{v}_p$. Weights may
> be any real numbers, including zero.

Deciding whether a specific $\mathbf{b}$ is such a combination reduces to a linear
system.

> **Worked example.** Determine whether $\mathbf{b}$ is a linear combination of
> $\mathbf{a}_1$ and $\mathbf{a}_2$, where
>
> $$
> \mathbf{a}_1 = \begin{bmatrix} 1 \\ -2 \\ -5 \end{bmatrix},\quad
> \mathbf{a}_2 = \begin{bmatrix} 2 \\ 5 \\ 6 \end{bmatrix},\quad
> \mathbf{b} = \begin{bmatrix} 7 \\ 4 \\ -3 \end{bmatrix}.
> $$
>
> The equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{b}$ gives the system
> with augmented matrix $[\,\mathbf{a}_1\ \mathbf{a}_2\ \mathbf{b}\,]$. Add $2$
> times row 1 to row 2 and $5$ times row 1 to row 3:
>
> $$
> \begin{bmatrix} 1 & 2 & 7 \\ -2 & 5 & 4 \\ -5 & 6 & -3 \end{bmatrix}
> \;\longrightarrow\;
> \begin{bmatrix} 1 & 2 & 7 \\ 0 & 9 & 18 \\ 0 & 16 & 32 \end{bmatrix}
> \;\longrightarrow\;
> \begin{bmatrix} 1 & 0 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{bmatrix}.
> $$
>
> The system is consistent with $x_1 = 3$, $x_2 = 2$, so
> $\mathbf{b} = 3\mathbf{a}_1 + 2\mathbf{a}_2$.

> **Theorem (Vector equation as a system).** The vector equation
> $$
> x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + \cdots + x_n \mathbf{a}_n = \mathbf{b}
> $$
> has the same solution set as the linear system whose augmented matrix is
> $[\,\mathbf{a}_1\ \mathbf{a}_2\ \cdots\ \mathbf{a}_n\ \mathbf{b}\,]$. In
> particular, $\mathbf{b}$ is a linear combination of $\mathbf{a}_1, \dots,
> \mathbf{a}_n$ if and only if that system is consistent.

The set of _all_ reachable combinations gets a name.

> **Definition (Span).** For $\mathbf{v}_1, \dots, \mathbf{v}_p$ in
> $\mathbb{R}^n$, $\operatorname{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ is
> the set of all their linear combinations. Asking whether $\mathbf{b} \in
> \operatorname{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ is asking whether
> $x_1\mathbf{v}_1 + \cdots + x_p\mathbf{v}_p = \mathbf{b}$ has a solution.

Every span contains $\mathbf{0}$ (take all weights zero) and every scalar multiple
of each $\mathbf{v}_i$.[^lay-13]

### The geometry of a span

In $\mathbb{R}^3$ a span has a shape that grows with the number of independent
directions it holds.

- **One nonzero vector.** $\operatorname{Span}\{\mathbf{v}\}$ is the set of all
  scalar multiples of $\mathbf{v}$: a line through the origin.
- **Two vectors, neither a multiple of the other.**
  $\operatorname{Span}\{\mathbf{u}, \mathbf{v}\}$ is the plane through the origin
  containing both.

$$
% caption: In R^3 the span of one vector is a line through 0; the span of two
% non-parallel vectors is the plane through 0 that contains both.
\begin{tikzpicture}[scale=1.0, font=\footnotesize, >=stealth]
\definecolor{acc}{HTML}{4A6FA5}
% --- left: span of one vector = line ---
\begin{scope}
  \draw[black, ->] (0,0) -- (2.4,0.5) node[right] {$x_1$};
  \draw[black, ->] (0,0) -- (0,2.6) node[above] {$x_3$};
  \draw[black, ->] (0,0) -- (1.3,0.9) node[right] {$x_2$};
  \draw[acc, thick] (-1.5,-1.35) -- (1.7,1.53);
  \draw[very thick, ->] (0,0) -- (1.0,0.9) node[above left] {$\mathbf{v}$};
  \node[acc] at (2.0,1.35) {Span of $\mathbf{v}$};
\end{scope}
% --- right: span of two vectors = plane ---
\begin{scope}[xshift=6.2cm]
  \draw[acc!55, fill=acc!8] (-1.6,-0.9) -- (1.9,-0.3) -- (1.6,1.7) -- (-1.9,1.1) -- cycle;
  \draw[black, ->] (0,-0.55) -- (0,2.4) node[above] {$x_3$};
  \draw[very thick, ->] (0,0.15) -- (1.3,0.6) node[below right] {$\mathbf{u}$};
  \draw[very thick, ->] (0,0.15) -- (-1.0,0.75) node[above left] {$\mathbf{v}$};
  \node[acc] at (1.3,1.4) {Span of $\mathbf{u}$, $\mathbf{v}$};
\end{scope}
\end{tikzpicture}
$$

A membership test remains a row reduction.

> **Worked example.** Decide whether $\mathbf{b}$ lies in
> $\operatorname{Span}\{\mathbf{a}_1, \mathbf{a}_2\}$, where
>
> $$
> \mathbf{a}_1 = \begin{bmatrix} 1 \\ -2 \\ 3 \end{bmatrix},\quad
> \mathbf{a}_2 = \begin{bmatrix} 5 \\ -13 \\ -3 \end{bmatrix},\quad
> \mathbf{b} = \begin{bmatrix} -3 \\ 8 \\ 1 \end{bmatrix}.
> $$
>
> Reduce $[\,\mathbf{a}_1\ \mathbf{a}_2\ \mathbf{b}\,]$. Add $2$ times row 1 to
> row 2 and $-3$ times row 1 to row 3, then subtract $6$ times the new row 2 from
> row 3:
>
> $$
> \begin{bmatrix} 1 & 5 & -3 \\ -2 & -13 & 8 \\ 3 & -3 & 1 \end{bmatrix}
> \;\longrightarrow\;
> \begin{bmatrix} 1 & 5 & -3 \\ 0 & -3 & 2 \\ 0 & -18 & 10 \end{bmatrix}
> \;\longrightarrow\;
> \begin{bmatrix} 1 & 5 & -3 \\ 0 & -3 & 2 \\ 0 & 0 & -2 \end{bmatrix}.
> $$
>
> The last row reads $0 = -2$, so the system is inconsistent and $\mathbf{b}$ is
> not in the plane $\operatorname{Span}\{\mathbf{a}_1, \mathbf{a}_2\}$.

$$
% caption: b lies in Span of a_1, a_2 exactly when it sits in their plane through
% 0; a target off that plane makes the system inconsistent.
\begin{tikzpicture}[scale=1.0, font=\footnotesize, >=stealth]
\definecolor{acc}{HTML}{4A6FA5}
\draw[acc!55, fill=acc!8] (-2.0,-1.0) -- (2.2,-0.5) -- (1.8,1.6) -- (-2.4,1.1) -- cycle;
\node[acc, anchor=south] at (-1.0,1.8) {Span of $\mathbf{a}_1$, $\mathbf{a}_2$};
\draw[very thick, ->] (0,0.1) -- (1.4,0.35);
\node at (1.7,0.3) {$\mathbf{a}_1$};
\draw[very thick, ->] (0,0.1) -- (-1.1,0.7);
\node at (-1.4,0.95) {$\mathbf{a}_2$};
\draw[black!70, very thick, ->] (0,0.1) -- (0.7,2.15) node[above right] {$\mathbf{b}$};
\draw[black, dashed] (0.7,2.15) -- (0.7,0.5);
\fill[black!70] (0.7,0.5) circle (1.4pt);
\node[black, anchor=west, font=\scriptsize] at (0.9,1.5) {outside the plane};
\end{tikzpicture}
$$

## The matrix equation $A\mathbf{x} = \mathbf{b}$

Packaging the vectors as columns of a matrix compresses the notation and defines
the matrix-vector product.

> **Definition (Matrix-vector product).** If $A$ has columns $\mathbf{a}_1, \dots,
> \mathbf{a}_n$ and $\mathbf{x} \in \mathbb{R}^n$, then
> $$
> A\mathbf{x} = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n \end{bmatrix}
> \begin{bmatrix} x_1 \\ \vdots \\ x_n \end{bmatrix}
> = x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + \cdots + x_n \mathbf{a}_n.
> $$
> The product is defined only when the number of columns of $A$ equals the number
> of entries in $\mathbf{x}$.

So $A\mathbf{x}$ is by definition a linear combination of the columns of $A$, with
the entries of $\mathbf{x}$ as weights. This definition ties the three
formulations together.[^lay-14]

> **Theorem (Three views of a system).** For an $m \times n$ matrix $A$ with
> columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ and $\mathbf{b} \in \mathbb{R}^m$,
> the matrix equation $A\mathbf{x} = \mathbf{b}$, the vector equation $x_1
> \mathbf{a}_1 + \cdots + x_n \mathbf{a}_n = \mathbf{b}$, and the linear system
> with augmented matrix $[\,\mathbf{a}_1\ \cdots\ \mathbf{a}_n\ \mathbf{b}\,]$ all
> have the same solution set.

$$
% caption: One problem, three equivalent forms; all are solved by row reducing
% the same augmented matrix.
\begin{tikzpicture}[scale=1.0, font=\footnotesize,
  b/.style={draw, minimum width=32mm, minimum height=13mm, align=center}]
\definecolor{acc}{HTML}{4A6FA5}
\node[b] (m) at (0,0) {matrix equation\\$A\mathbf{x}=\mathbf{b}$};
\node[b] (v) at (4.4,0) {vector equation\\$x_1\mathbf{a}_1 + x_2\mathbf{a}_2 = \mathbf{b}$};
\node[b] (s) at (8.8,0) {linear system\\(the equations)};
\node[b, draw=acc, text=acc, thick] (r) at (4.4,-2.2) {row reduce\\$[\,A \;\; \mathbf{b}\,]$};
\draw[black] (m) -- (v); \draw[black] (v) -- (s);
\draw[->] (m) -- (r); \draw[->] (v) -- (r); \draw[->] (s) -- (r);
\end{tikzpicture}
$$

A worked case: writing the combination $3\mathbf{v}_1 - 5\mathbf{v}_2 +
7\mathbf{v}_3$ as $A\mathbf{x}$ just places the $\mathbf{v}_i$ into the columns of
$A$ and the weights into $\mathbf{x} = (3, -5, 7)$.

### Computing $A\mathbf{x}$: the row-vector rule

The column definition explains _what_ $A\mathbf{x}$ is; for hand computation the
row rule is faster.

> **Rule (Row-vector product).** If $A\mathbf{x}$ is defined, its $i$-th entry is
> the sum of the products of corresponding entries from row $i$ of $A$ and from
> $\mathbf{x}$.

For example,

$$
\begin{bmatrix} 2 & 3 & 4 \\ -1 & 5 & -3 \\ 6 & -2 & 8 \end{bmatrix}
\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}
= \begin{bmatrix} 2x_1 + 3x_2 + 4x_3 \\ -x_1 + 5x_2 - 3x_3 \\ 6x_1 - 2x_2 + 8x_3 \end{bmatrix}.
$$

The **identity matrix** $I_n$ (ones on the diagonal, zeros elsewhere) satisfies
$I_n\mathbf{x} = \mathbf{x}$ for every $\mathbf{x}$, because each row picks off one
entry. Two algebraic properties follow directly from the definition and will be
used constantly.

> **Theorem (Linearity of $\mathbf{x} \mapsto A\mathbf{x}$).** For an $m \times n$
> matrix $A$, vectors $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$, and scalar $c$:
> $$
> A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}, \qquad A(c\mathbf{u}) = c(A\mathbf{u}).
> $$

Both follow by writing each side as a linear combination of columns and using the
entrywise rules of $\mathbb{R}^n$. These two identities underlie
[linear transformations](/linear-algebra/linear-systems/linear-transformations),
where $\mathbf{x} \mapsto A\mathbf{x}$ becomes the object of study.

## Existence: one $\mathbf{b}$ versus every $\mathbf{b}$

Two existence questions sit side by side, and they are not the same.

**Is $A\mathbf{x} = \mathbf{b}$ consistent for one given $\mathbf{b}$?** By the
definition of $A\mathbf{x}$, this holds if and only if $\mathbf{b}$ is a linear
combination of the columns of $A$ — the span membership test, decided by row
reducing $[\,A\ \mathbf{b}\,]$.

**Is $A\mathbf{x} = \mathbf{b}$ consistent for _every_ $\mathbf{b}$ in
$\mathbb{R}^m$?** This is stronger, and it does not depend on any particular
$\mathbf{b}$. It depends only on the pivot pattern of $A$ itself.

> **Theorem (Columns spanning $\mathbb{R}^m$).** For an $m \times n$ matrix $A$,
> the following are equivalent — all true or all false:
> - **(a)** $A\mathbf{x} = \mathbf{b}$ has a solution for each $\mathbf{b} \in \mathbb{R}^m$;
> - **(b)** each $\mathbf{b} \in \mathbb{R}^m$ is a linear combination of the columns of $A$;
> - **(c)** the columns of $A$ span $\mathbb{R}^m$;
> - **(d)** $A$ has a pivot position in every row.

Statements (a), (b), (c) are the same fact worded three ways; (d) connects it to
the pivot pattern. If $A$ has a pivot in every row, then no echelon form of
$[\,A\ \mathbf{b}\,]$ can carry a row $[\,0\ \cdots\ 0 \mid 1\,]$ — every row
already spends its pivot on a column of $A$ — so the system is consistent
whatever $\mathbf{b}$ is. If some row of the echelon form of $A$ is all zeros, a
$\mathbf{b}$ can be chosen to put a $1$ there, forcing inconsistency.[^lay-14proof]

> **Warning.** The pivot condition is on $A$, the coefficient matrix, not the
> augmented matrix $[\,A\ \mathbf{b}\,]$. An augmented matrix with a pivot in
> every row tells you nothing about consistency by itself.

### A description of the reachable $\mathbf{b}$

When the columns do _not_ span $\mathbb{R}^m$, the set of reachable $\mathbf{b}$
is itself describable.

> **Worked example.** Describe the vectors $\mathbf{b}$ for which
> $A\mathbf{x} = \mathbf{b}$ is consistent, where
>
> $$
> A = \begin{bmatrix} 1 & 3 & 4 \\ -4 & 2 & -6 \\ -3 & -2 & -7 \end{bmatrix}.
> $$
>
> Row reduce $[\,A\ \mathbf{b}\,]$ with the constants kept symbolic. Add $4$ times
> row 1 to row 2 and $3$ times row 1 to row 3, then subtract $\tfrac12$ times the
> new row 2 from row 3:
>
> $$
> \begin{bmatrix} 1 & 3 & 4 & b_1 \\ 0 & 14 & 10 & b_2 + 4b_1 \\ 0 & 7 & 5 & b_3 + 3b_1 \end{bmatrix}
> \;\longrightarrow\;
> \begin{bmatrix} 1 & 3 & 4 & b_1 \\ 0 & 14 & 10 & b_2 + 4b_1 \\ 0 & 0 & 0 & b_1 - \tfrac12 b_2 + b_3 \end{bmatrix}.
> $$
>
> The bottom row forces $0 = b_1 - \tfrac12 b_2 + b_3$, so the system is
> consistent exactly when
>
> $$
> b_1 - \tfrac12 b_2 + b_3 = 0,
> $$
>
> a plane through the origin in $\mathbb{R}^3$. This plane is
> $\operatorname{Span}\{\mathbf{a}_1, \mathbf{a}_2, \mathbf{a}_3\}$, which falls
> short of all of $\mathbb{R}^3$ because $A$ has only two pivots.

| Question | Depends on | Test |
| --- | --- | --- |
| $A\mathbf{x} = \mathbf{b}$ solvable for this $\mathbf{b}$? | $A$ and $\mathbf{b}$ | reduce $[\,A\ \mathbf{b}\,]$; no pivot in last column |
| $A\mathbf{x} = \mathbf{b}$ solvable for every $\mathbf{b}$? | $A$ alone | reduce $A$; a pivot in every row |
| Which $\mathbf{b}$ are reachable? | $A$ | span of the columns of $A$ |

[^lay-13]: **Lay**, _Linear Algebra and Its Applications_, §1.3 — Vector Equations: vectors in $\mathbb{R}^n$, the parallelogram rule, linear combinations, span, and the equivalence between a vector equation and the linear system with the vectors as columns.
[^lay-14]: **Lay**, §1.4 — The Matrix Equation $A\mathbf{x} = \mathbf{b}$: the definition of $A\mathbf{x}$ as a linear combination of columns, the row-vector rule, and the three equivalent formulations of a linear system.
[^lay-14proof]: **Lay**, §1.4, Theorem 4 and its proof: the equivalence of the four spanning statements, resting on the fact that $A$ has a pivot in every row iff no echelon form of $[\,A\ \mathbf{b}\,]$ has an inconsistent last row.