Matrix Operations

A linear transformation $x \mapsto A x$ is stored as a matrix; composing two such maps produces a third, and matrix multiplication is the arithmetic that computes its matrix. The single requirement $A (B x) = (A B) x$ fixes every rule that follows: why the product is defined column by column, why it is associative but not commutative, and why its size constraints are what they are. Addition and scalar multiplication are the entrywise operations carried over from vectors.

Notation and entries

An $m \times n$ matrix $A$ has $m$ rows and $n$ columns. The scalar in row $i$ and column $j$ is the $(i, j)$ -entry $a_{ij}$ , and $A = [a_{ij}]$ . Each column is a vector in $R^{m}$ , so writing the columns as $a_{1}, \dots, a_{n}$ gives

A = [a_{1} a_{2} \dots a_{n}], a_{ij} = (i th entry of a_{j}) .

Several kinds of matrix recur often enough to name:

Diagonal entries: the entries $a_{11}, a_{22}, a_{33}, \dots$ ; together they form the main diagonal.
Diagonal matrix: a square matrix whose off-diagonal entries are all zero.
Identity matrix $I_{n}$ : the diagonal matrix with $1$ s on the diagonal, so $I_{m} x = x$ for every $x$ in $R^{m}$ .
Zero matrix $0$ : every entry zero; its size is read from context.

Sums and scalar multiples

Two matrices are equal when they have the same size and the same corresponding entries. If $A$ and $B$ are both $m \times n$ , the sum $A + B$ is the $m \times n$ matrix formed by adding corresponding entries; $A + B$ is undefined when the sizes differ. For a scalar $r$ , the scalar multiple $r A$ multiplies every entry by $r$ , and $- A$ means $(- 1) A$ , with $A - B = A + (- 1) B$ .

For example, with

A = [4 - 1 0352], B = [131517],

we get $A + B = [521869]$ and $2 B = [26210214]$ , so $A - 2 B = [2 - 7 - 2 - 7 3 - 12]$ .

Because addition and scaling reduce to the corresponding operations on the column vectors, matrices inherit all the familiar algebraic laws.¹

Multiplication as composition

Multiplying a vector $x$ by $B$ produces $B x$ ; multiplying that in turn by $A$ produces $A (B x)$ . The composite map $x \mapsto A (B x)$ is again linear, and the product $A B$ is defined to be its matrix, so that

A (B x) = (A B) x for all x .

The product AB is the single matrix whose action reproduces "multiply by B, then by A" — matrix multiplication is composition of the two maps.

To turn this into a formula, take $B$ with columns $b_{1}, \dots, b_{p}$ and expand $B x = x_{1} b_{1} + \dots + x_{p} b_{p}$ . Linearity of multiplication by $A$ gives

A (B x) = x_{1} (A b_{1}) + \dots + x_{p} (A b_{p}) = [A b_{1} A b_{2} \dots A b_{p}] x .

The bracketed matrix is the product we set out to build.

Two consequences of the definition:

The number of columns of $A$ must equal the number of rows of $B$ (the inner dimensions match), or $A b_{j}$ is undefined.
$A B$ has as many rows as $A$ and as many columns as $B$ (the outer dimensions).

Size bookkeeping for AB: the inner dimensions must agree, and the outer dimensions become the size of the product.

For example, if $A$ is $3 \times 5$ and $B$ is $5 \times 2$ , then $A B$ is $3 \times 2$ , while $B A$ is undefined because the $2$ columns of $B$ do not match the $3$ rows of $A$ .

The row-column rule

The column definition is right for theory, but individual entries are computed faster by pairing a row of $A$ with a column of $B$ .

The entry in position $(i, j)$ is the dot product of row $i$ of $A$ with column $j$ of $B$ . A related identity records what a whole row of the product is: $row_{i} (A B) = row_{i} (A) B$ .

The (i,j)-entry of AB pairs the ith row of A with the jth column of B and sums the products of matched entries.

The algebra of products

Because matrix multiplication is composition of functions, and composition of functions is associative, products obey the same grouping and distribution rules as ordinary arithmetic, with one omission: commutativity.²

Associativity lets parentheses move freely, so a product $A B C D$ is unambiguous as long as the left-to-right order is preserved. That order is where matrix algebra departs from scalar algebra.

Non-commutativity is visible in the columns: the columns of $A B$ are combinations of the columns of $A$ , whereas the columns of $B A$ are built from the columns of $B$ , so the two products describe different maps. When $A B = B A$ does hold, $A$ and $B$ are said to commute.

Powers

For a square $n \times n$ matrix $A$ and a positive integer $k$ , the power $A^{k}$ is the product of $k$ copies of $A$ :

A^{k} = k AA \dots A, A^{0} = I .

Setting $A^{0} = I$ keeps $A^{k} x$ meaning apply $A$ to $x$ a total of $k$ times, with zero applications leaving $x$ unchanged. Powers of matrices drive the Leontief model, Markov chains, and the study of eigenvalues, where a good factorization of $A$ turns $A^{k}$ into something computable.

The transpose

Rows and columns exchange roles under the transpose.

The transpose reflects a matrix across its main diagonal; the (i,j) entry lands in position (j,i).

The transpose interacts with the other operations through four rules; the last reverses the order of a product.³

In words, the transpose of a product is the product of the transposes in reverse order, and this extends to any number of factors: $(A B C)^{⊤} = C^{⊤} B^{⊤} A^{⊤}$ . In general $(A B)^{⊤} \neq = A^{⊤} B^{⊤}$ , even when the right side happens to be defined.

Inner and outer products

Viewing a vector in $R^{n}$ as an $n \times 1$ matrix turns two special transpose products into familiar objects. For $u, v \in R^{n}$ :

Inner product $u^{⊤} v$ : a $1 \times 1$ matrix, written as the single real number $u_{1} v_{1} + \dots + u_{n} v_{n}$ . This is the dot product that underlies length and orthogonality.
Outer product $u v^{⊤}$ : an $n \times n$ matrix whose $(i, j)$ -entry is $u_{i} v_{j}$ . Outer products reappear as the rank-one pieces of block multiplication and the SVD.

Operation	Requirement	Result size	Reads as
$A + B$	same size	$m \times n$	entrywise sum
$r A$	any	$m \times n$	entrywise scale
$A B$	cols $A$ = rows $B$	$m \times p$	composition of maps
$A^{⊤}$	any	$n \times m$	reflect across diagonal
$u^{⊤} v$	same length	$1 \times 1$	scalar (dot product)
$u v^{⊤}$	any lengths	$n \times m$	rank-one matrix

Numerical note

High-performance libraries such as LAPACK compute $A B$ one column at a time, in agreement with the column definition, because that layout matches how matrices are stored in memory and because the columns of $A B$ can be assigned to separate processors and computed in parallel. The order of a chained product also matters for cost, not just correctness: when $B$ is square and $C$ has few columns, computing $A (B C)$ can be far cheaper than $(A B) C$ , even though the two agree by associativity. The LU factorization turns these observations into a systematic way to solve many systems at once.

Lay, Linear Algebra and Its Applications, §2.1 — Theorem 1: the commutative, associative, and distributive laws for matrix sums and scalar multiples, each verified column by column from the vector case. ↩
Lay, §2.1 — Theorem 2: associativity of multiplication follows from associativity of function composition, together with the two distributive laws, $r (A B) = (r A) B = A (r B)$ , and $I_{m} A = A = A I_{n}$ ; the accompanying warnings collect the three ways matrix algebra differs from scalar algebra. ↩
Lay, §2.1 — Theorem 3 and the definition of the transpose, including $(A B)^{⊤} = B^{⊤} A^{⊤}$ and the inner/outer product interpretations of $u^{⊤} v$ and $u v^{⊤}$ . ↩