Inner Product Spaces

Every geometric fact so far followed from the four properties of the dot product, never from the coordinate formula $\sum u_i{v}_i$. Promoting those properties to axioms defines an inner product on any vector space, and the apparatus — length, distance, orthogonality, projection, Gram–Schmidt, best approximation — transfers unchanged. The spaces can then hold polynomials or continuous functions, and least-squares approximation becomes function approximation.

The axioms

The list restates the conclusions of the inner-product-properties theorem for the dot product, now taken as requirements. Any rule satisfying them supports the full geometric vocabulary.

Weighted inner product on ${\mathbb{R}}^n$. Fix positive weights $w_1,\dots ,w_n$ and set $⟨\mathbf{u},\mathbf{v}⟩=w_1{u}_1{v}_1+\cdots +w_n{u}_n{v}_n$. Each axiom checks directly, so this is an inner product. Larger weights emphasize the more reliable coordinates, the basis of weighted least squares.

Evaluation inner product on $P_n$. Fix distinct reals $t_0,\dots ,t_n$. For polynomials $p,q$ of degree at most $n$, set

The first three axioms are immediate. For the fourth, $⟨p,p⟩=\sum p(t_i{)}^2\ge 0$, and it vanishes only if $p$ is zero at $n+1$ points — which forces the degree-$\le n$ polynomial $p$ to be identically zero.

Length, distance, orthogonality

The definitions copy over verbatim, with $⟨\cdot ,\cdot ⟩$ in place of the dot product.

• Length: $\Vert \mathbf{v}\Vert =\sqrt{⟨\mathbf{v},\mathbf{v}⟩}$.
• Distance: $\mathrm{dist}(\mathbf{u},\mathbf{v})=\Vert \mathbf{u}-\mathbf{v}\Vert$.
• Orthogonality: $\mathbf{u}\perp \mathbf{v}$ means $⟨\mathbf{u},\mathbf{v}⟩=0$.

Because the axioms match, the earlier theorems hold without change. Gram–Schmidt still produces orthogonal bases of finite-dimensional subspaces, and the orthogonal projection onto a subspace $W$ with orthogonal basis $\{v_1,\dots ,v_p\}$ is

which remains the best approximation to $\mathbf{u}$ by elements of $W$.

Orthogonal polynomials

These are the orthogonal polynomials used in the statistical trend analysis of evenly spaced data.

Best approximation of functions

A recurring applied problem is to approximate a function $f$ by a simpler function $g$ drawn from a subspace $W$. When closeness is measured by an inner product, the best $g$ is the orthogonal projection of $f$ onto $W$.

Two inequalities

The Pythagorean decomposition of the figure, $\Vert \mathbf{v}{\Vert }^2=\Vert {\mathrm{proj}}_W\mathbf{v}{\Vert }^2+\Vert \mathbf{v}-{\mathrm{proj}}_W\mathbf{v}{\Vert }^2$, shows a projection is never longer than the vector itself. That single observation yields two central inequalities.

If $\mathbf{u}=0$ both sides vanish. Otherwise let $W=\mathrm{Span}\{\mathbf{u}\}$. The projection of $\mathbf{v}$ onto $W$ has length

and since $\Vert {\mathrm{proj}}_W\mathbf{v}\Vert \le \Vert \mathbf{v}\Vert$, rearranging gives the bound.

Expand and apply Cauchy–Schwarz:

and take square roots.

An inner product from an integral

The most widely used inner product space of analysis is $C[a,b]$, the continuous functions on an interval, with an inner product built by letting the evaluation sum become a Riemann integral.

The axioms follow from elementary properties of the definite integral; positivity uses that a continuous nonnegative integrand with zero integral must be identically zero. Length is then $\Vert f\Vert =\sqrt{{\int }_a^{b}f(t{)}^2\mathbf{d}t}$, and two functions are orthogonal when the integral of their product is zero.

Fourier approximation

The integral inner product on $C[0,2\pi ]$ makes the trigonometric functions orthogonal: for positive integers $m\ne n$,

and the same holds for the sine pairs and every sine-cosine pair. So

is an orthogonal set. Let $W$ be its span. The best approximation to a function $f$ by elements of $W$ is the projection onto $W$, called the $n$th-order Fourier approximation.

Each coefficient is the projection weight $⟨f,\cos kt⟩/⟨\cos kt,\cos kt⟩$, using $\Vert \cos kt{\Vert }^2=\Vert \sin kt{\Vert }^2=\pi$. The constant term is written $a_0/2$ so that the same formula for $a_k$ works at $k=0$.

Summary

One set of axioms carries the geometry of ${\mathbb{R}}^n$ into spaces of polynomials and functions. Cauchy–Schwarz and the triangle inequality hold in all of them, and least-squares approximation becomes best approximation by projection: orthogonal polynomials for trend analysis, Fourier series for signals.¹²