Biostatistics · Intermediate

Orthonormal Sets

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Orthonormal sets combine orthogonality and unit length, enabling simplified coordinate calculation via inner products and norm preservation through Parseval’s Formula. These properties streamline orthogonal projections onto subspaces, ensuring the projection is the unique closest vector while preserving geometric invariants like length and angle.

Intuition

Think of an orthonormal basis as a perfectly calibrated coordinate system where every axis is perpendicular to the others and has a standard unit length. This calibration removes the need for complex algebraic solving; to find a vector's coordinate along any axis, you simply project it onto that axis using an inner product. Because the axes are orthogonal, projecting onto one doesn't interfere with the others. This independence means that the total 'energy' or magnitude of a vector is simply the sum of the squares of its coordinates, a concept known as Parseval's Formula. When projecting a vector onto a subspace defined by such a basis, the result is the closest possible point in that subspace, and the operation never increases the vector's length.

Key equations

⟨ui,uj⟩=δij
(1)

Definition of orthonormality: inner product is 1 if indices match, 0 otherwise.

ui=vi‖vi‖
(2)

Normalization process to convert an orthogonal vector to unit length.

ci=⟨v,ui⟩
(3)

Coefficient calculation for a vector in an orthonormal basis.

‖v‖2=∑i=1n|ci|2
(4)

Parseval’s Formula relating vector norm to coordinate magnitudes.

p=∑i=1n⟨x,ui⟩ui
(5)

Orthogonal projection of vector x onto subspace S with orthonormal basis.

P=UUT
(6)

Matrix representation of the orthogonal projection operator.

Derivation

  1. Start with an orthogonal set of nonzero vectors {v_i}.
  2. Normalize each vector by dividing by its norm to create an orthonormal set {u_i} where <u_i, u_j> = delta_ij.
  3. Prove linear independence: If sum(c_i * v_i) = 0, taking the inner product with v_j isolates c_j * ||v_j||^2 = 0, forcing c_j = 0.
  4. Derive coordinate formula: For v = sum(c_i * u_i), take inner product with u_j. Orthogonality eliminates all terms except c_j * <u_j, u_j> = c_j.
  5. Establish Parseval's Formula: ||v||^2 = <v, v>. Substitute expansion and use orthonormality to sum |c_i|^2.
  6. Define projection p onto subspace S spanned by {u_i} as sum(<x, u_i> * u_i).
  7. Show p minimizes distance: The error vector (x - p) is orthogonal to S, making p the unique closest vector by the Pythagorean theorem.

Worked example

Problem

Find the orthogonal projection of vector 𝐰=(5,3,4)T onto the subspace S in ℝ3 consisting of vectors (x,y,0)T.

  1. Identify the orthonormal basis for S: u_1 = (1, 0, 0)^T and u_2 = (0, 1, 0)^T.
  2. Calculate coefficient c_1 = <w, u_1> = 5*1 + 3*0 + 4*0 = 5.
  3. Calculate coefficient c_2 = <w, u_2> = 5*0 + 3*1 + 4*0 = 3.
  4. Construct projection p = c_1*u_1 + c_2*u_2 = 5*(1,0,0)^T + 3*(0,1,0)^T = (5, 3, 0)^T.

Answer p = (5, 3, 0)^T

Common mistakes

  • Applying c_i = <v, u_i> to an orthogonal but non-normalized basis; the formula requires unit length vectors.
  • Assuming the projection matrix P depends on the specific orthonormal basis chosen; P is unique to the subspace.
  • Confusing norm preservation: ||p|| <= ||v|| generally, with equality only if v is already in S.
  • Forgetting that orthogonal matrices preserve inner products and norms, meaning they represent rigid rotations/reflections.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

This is a text lesson: it has no animation code yet.

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.