Biostatistics · Advanced

Orthogonal Polynomials

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Orthogonal polynomials form a basis for function spaces where distinct polynomials have zero inner product, enabling efficient least squares approximation without solving normal equations. They are constructed via Gram-Schmidt orthogonalization and satisfy a three-term recurrence relation, with classical examples like Legendre and Chebyshev polynomials defined by specific weight functions. Their zeros provide optimal nodes for Gaussian quadrature, ensuring exact integration for polynomials up to degree 2n−1.

Intuition

Think of orthogonal polynomials as a specialized coordinate system for functions, similar to how orthogonal vectors simplify geometry in linear algebra. In this system, each polynomial pn(x) is 'perpendicular' to all lower-degree polynomials, meaning they capture independent aspects of a function's shape. This independence allows us to approximate complex functions by simply projecting them onto these basis elements, avoiding the computational complexity of coupled equations. The structure is rigidly defined by a weight function that dictates the 'geometry' of the space, ensuring that the polynomials remain orthogonal across the chosen interval.

Key equations

⟨pi,pj⟩=∫abpi(x)pj(x)w(x)dx=0for i≠j
(1)

Definition of orthogonality for distinct polynomials in the sequence.

p(x)=∑i=0n−1⟨f,pi⟩⟨pi,pi⟩pi(x)
(2)

Best least squares approximation of function f using orthogonal basis.

αn+1pn+1(x)=(x−βn+1)pn(x)−αnγnpn−1(x)
(3)

Three-term recurrence relation for generating orthogonal polynomials.

∫abf(x)w(x)dx≈∑i=1nAif(xi)
(4)

Gaussian quadrature formula using zeros of pn(x) as nodes.

Derivation

  1. Start with the monomial basis {1,x,x2,…} in the space of polynomials.
  2. Apply Gram-Schmidt orthogonalization to generate pn(x) orthogonal to all p0,…,pn−1.
  3. Since pn has degree n and is orthogonal to the subspace Pn−1, it satisfies a three-term recurrence relation derived from expanding xpn(x) in the orthogonal basis.
  4. The coefficients of this recurrence depend on the leading coefficients and norms of the polynomials.
  5. For least squares approximation, the orthogonality property ensures that the projection coefficients are independent, simplifying the solution to a sum of projections.
  6. The zeros of pn(x) are real, distinct, and lie in (a,b), allowing Gaussian quadrature to achieve exactness for polynomials of degree up to 2n−1.

Worked example

Problem

Calculate the first two Legendre polynomials P0(x) and P1(x) on [−1,1] with weight w(x)=1, given the recurrence (n+1)Pn+1(x)=(2n+1)xPn(x)−nPn−1(x) and normalization Pn(1)=1.

  1. For n=0, the recurrence is not directly applicable for generation, but we define P0(x) based on normalization. Since P0(1)=1 and degree is 0, P0(x)=1.
  2. For n=1, use the recurrence with n=0: (0+1)P1(x)=(2(0)+1)xP0(x)−0⋅P−1(x).
  3. Simplify: P1(x)=1⋅x⋅1−0=x.
  4. Check normalization: P1(1)=1, which satisfies the condition.

Answer P_0(x) = 1, P_1(x) = x

Common mistakes

  • Confusing orthogonality with orthonormality; orthogonality requires zero inner product for distinct polynomials, but orthonormality also requires unit norm.
  • Assuming the leading coefficient an is fixed; it can be chosen arbitrarily (e.g., an=1) without affecting orthogonality, but it changes the normalization.
  • Believing Gaussian quadrature is exact for all polynomials; it is exact only for polynomials of degree less than 2n when using n nodes.
  • Ignoring the weight function w(x); the orthogonality and recurrence relations depend critically on the specific weight function and interval.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

This is a text lesson: it has no animation code yet.

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.