Content
Summary
Solving the linear system Ax = b isolates the unknown vector x by left-multiplying both sides by the inverse of matrix A, resulting in x = A⁻¹b. This process is fundamental for finding regression weights in least-squares problems, though numerical stability issues may require using pseudo-inverses or other methods instead of direct inversion.
Intuition
Think of solving Ax = b as trying to isolate x, similar to how you would solve the scalar equation ax = b by dividing by a. In matrix algebra, you cannot simply 'divide' by A; instead, you must multiply by its inverse, A⁻¹, on the left side. This operation cancels out A on the left side because A⁻¹A equals the identity matrix I, leaving x alone. However, this straightforward approach assumes A is invertible and well-behaved. If the matrix is poorly conditioned or not square, the inverse might not exist or might yield inaccurate results, necessitating the use of a pseudo-inverse to find the best approximation.
Key equations
The definition of the inverse matrix A⁻¹, where I is the identity matrix.
The scalar analogy for solving linear equations, where multiplying by a⁻¹ isolates x.
The matrix solution for x, obtained by left-multiplying both sides by A⁻¹.
Derivation
- Start with the linear system equation Ax = b.
- Left-multiply both sides of the equation by the inverse of A, denoted as A⁻¹.
- This yields A⁻¹Ax = A⁻¹b.
- Recognize that A⁻¹A simplifies to the identity matrix I, based on the definition AA⁻¹ = I.
- The equation becomes Ix = A⁻¹b.
- Since Ix = x, the final solution is x = A⁻¹b.
Worked example
Given a system Ax = b where A is a square invertible matrix, explain the step to isolate x and verify the inverse.
- Identify that to isolate x, we must remove A from the left side.
- Multiply both sides by A⁻¹ on the left: A⁻¹(Ax) = A⁻¹b.
- Simplify A⁻¹A to I, resulting in x = A⁻¹b.
- Verify the inverse by computing AA⁻¹; the result must be the identity matrix I.
Answer The solution is x = A⁻¹b, verified by ensuring AA⁻¹ = I.
Common mistakes
- Assuming that a matrix inverse always exists or produces a perfectly accurate solution.
- Confusing the scalar division operation with matrix multiplication, noting that order matters (left-multiplication).
- Believing that the pseudo-inverse is only for non-square matrices, whereas it is also used for poorly behaved square matrices.