The Inverse and the Invertible Matrix Theorem

The reciprocal $5^{-1}$ undoes multiplication by $5$: $5^{-1}\cdot 5=1$. A square matrix has an analogous partner, defined so that multiplying by it undoes multiplication by $A$. Two features of matrices constrain the generalization: multiplication is not commutative, so both orders $CA=I$ and $AC=I$ must be required, and only square matrices can have a full two-sided inverse.

Definition and uniqueness

The inverse is unique. If $B$ and $C$ were both inverses, then

so there is only one inverse, written $A^{-1}$, with $A{A}^{-1}=A^{-1}A=I$. A matrix with no inverse is called singular; an invertible matrix is nonsingular.

The 2×2 formula

For $2\times 2$ matrices there is a closed form, controlled by a single number.

The quantity $ad-bc$ is the determinant of $A$, written $\det A$; the $2\times 2$ inverse criterion says a $2\times 2$ matrix is invertible exactly when $\det A\ne 0$. The determinant of a general square matrix extends this test to every size. For $A=\left[\begin{array}{cc}3& 4\\ 5& 6\end{array}\right]$, $\det A=18-20=-2\ne 0$, so

Solving $Ax=b$

An inverse turns a square system into a single multiplication.

The solution exists because $A(A^{-1}b)=(A{A}^{-1})b=Ib=b$, and it is unique because $Au=b$ forces $u=A^{-1}Au=A^{-1}b$. In practice $A^{-1}b$ is rarely the way to solve a system numerically — row reducing $[A\mid b]$ is faster and more accurate — but the unique-solution theorem is the theoretical bridge between inverses and solvability, and for hand computation on a $2\times 2$ system the formula is convenient.

Algebra of inverses

Three rules describe how the inverse interacts with itself, with products, and with the transpose. The product rule reverses order, exactly as the transpose does.¹

The product rule is proved by checking the definition: $(AB)(B^{-1}{A}^{-1})=A(B{B}^{-1})A^{-1}=AI{A}^{-1}=I$, and similarly on the other side. It generalizes: a product of invertible matrices is invertible, and the inverse is the product of the inverses in reverse order, $(A_1{A}_2\cdots A_k{)}^{-1}=A_k^{-1}\cdots A_2^{-1}{A}_1^{-1}$.

Elementary matrices

Computing $A^{-1}$ efficiently rests on a connection between inverses and row operations, made through elementary matrices.

Left-multiplying any matrix $A$ by an elementary matrix $E$ performs that same row operation on $A$: if $E$ is $I$ with rows $1$ and $2$ swapped, then $EA$ is $A$ with rows $1$ and $2$ swapped. Because row operations are reversible, every elementary matrix is invertible, and its inverse is the elementary matrix of the same type that undoes the operation.

For instance, the operation add $4$ times row $1$ to row $3$ applied to $I_3$ gives the elementary matrix $E$, and left-multiplying $A$ by $E$ performs that same operation on $A$:

An elementary row operation carried out on an $m\times n$ matrix $A$ produces $EA$, where $E$ is $I_m$ with that same operation applied. Since each step of a row reduction is a left-multiplication by an elementary matrix, reducing $A$ to $I$ amounts to a product $E_p\cdots E_{1}A=I$.

Row reduction finds the inverse

The reason is direct. If $A$ is invertible, $Ax=b$ is solvable for every $b$, so $A$ has a pivot in every row; being square, its reduced echelon form is $I_n$. Conversely, if $A$ reduces to $I_n$, then $E_p\cdots E_{1}A=I_n$, so $A=(E_p\cdots E_1{)}^{-1}$ is invertible, and $A^{-1}=E_p\cdots E_1$ is precisely what those same operations do to $I_n$.

Placing $A$ and $I$ side by side lets a single row reduction act on both.

If instead a row of zeros had appeared in the left block, the reduction would stop: $A$ is not row equivalent to $I$, hence singular.

Another view: columns solve $Ax=e_j$

Writing $I=[e_1\cdots e_n]$, the reduction of $[A\mid I]$ solves the $n$ systems $Ax=e_1,\dots ,Ax=e_n$ simultaneously. Since $A{A}^{-1}=I$, column $j$ of $A^{-1}$ solves $Ax=e_j$. When an application needs only one or two columns of the inverse, only the corresponding systems must be solved.

The Invertible Matrix Theorem

Row reduction, linear independence, spanning, and one-to-one/onto maps all turn out to be the same condition on a square matrix, stated in different vocabularies. The Invertible Matrix Theorem collects them.²

The proof runs a circle of implications $\text{(a)}\Rightarrow \text{(j)}\Rightarrow \text{(d)}\Rightarrow \text{(c)}\Rightarrow \text{(b)}\Rightarrow \text{(a)}$, then links the remaining statements to that circle through results already available for any matrix: (d), (e), (f) are equivalent for every matrix, as are (g), (h), (i).

Two cautions. Statement (g) can be strengthened to $Ax=b$ has a unique solution for each $b$ by the unique-solution theorem, so uniqueness is built in. And the theorem applies only to square matrices: if the columns of a $4\times 3$ matrix are linearly independent, nothing follows about the solvability of $Ax=b$, because that matrix is not square.

A companion fact drops out of the theorem: for square $A$ and $B$, a one-sided identity is enough. If $AB=I$, then $A$ and $B$ are both invertible, with $B=A^{-1}$ and $A=B^{-1}$.

Invertible linear transformations

The theorem transfers to maps. A linear transformation $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is invertible if there is a function $S:{\mathbb{R}}^n\to {\mathbb{R}}^n$ with $S(T(x))=x$ and $T(S(x))=x$ for all $x$. When $A$ is the standard matrix of $T$, invertibility of the map and of the matrix coincide.

A one-to-one linear map from ${\mathbb{R}}^n$ into ${\mathbb{R}}^n$ therefore has independent columns in its standard matrix (statement (e)), so the matrix is invertible, the map is automatically onto (statement (i)), and $T$ has an inverse $T^{-1}(x)=A^{-1}x$. For square maps, one-to-one and onto are equivalent: each implies the other.

Numerical note

In floating-point arithmetic a matrix can be ill-conditioned: invertible in exact arithmetic, yet so close to singular that small perturbations of its entries make it singular. Row reduction may then report fewer than $n$ pivots because of roundoff, or make a genuinely singular matrix look invertible. Matrix software reports a condition number measuring this sensitivity: it is $1$ for $I$, grows without bound as a matrix approaches singularity, and is infinite for a singular matrix. The singular value decomposition gives the reliable way to judge how close a matrix is to losing rank, and the numerical module develops the condition number in full.