AI, machine learning and deep learning · Intermediate

Solving Ax = b

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Solving the linear system Ax = b isolates the unknown vector x by left-multiplying both sides by the inverse of matrix A, resulting in x = A⁻¹b. This process is fundamental for finding regression weights in least-squares problems, though numerical stability issues may require using pseudo-inverses or other methods instead of direct inversion.

Intuition

Think of solving Ax = b as trying to isolate x, similar to how you would solve the scalar equation ax = b by dividing by a. In matrix algebra, you cannot simply 'divide' by A; instead, you must multiply by its inverse, A⁻¹, on the left side. This operation cancels out A on the left side because A⁻¹A equals the identity matrix I, leaving x alone. However, this straightforward approach assumes A is invertible and well-behaved. If the matrix is poorly conditioned or not square, the inverse might not exist or might yield inaccurate results, necessitating the use of a pseudo-inverse to find the best approximation.

Key equations

AA−1=I
(1)

The definition of the inverse matrix A⁻¹, where I is the identity matrix.

ax=b⟹x=b/a
(2)

The scalar analogy for solving linear equations, where multiplying by a⁻¹ isolates x.

Ax=b⟹x=A−1b
(3)

The matrix solution for x, obtained by left-multiplying both sides by A⁻¹.

Derivation

  1. Start with the linear system equation Ax = b.
  2. Left-multiply both sides of the equation by the inverse of A, denoted as A⁻¹.
  3. This yields A⁻¹Ax = A⁻¹b.
  4. Recognize that A⁻¹A simplifies to the identity matrix I, based on the definition AA⁻¹ = I.
  5. The equation becomes Ix = A⁻¹b.
  6. Since Ix = x, the final solution is x = A⁻¹b.

Worked example

Problem

Given a system Ax = b where A is a square invertible matrix, explain the step to isolate x and verify the inverse.

  1. Identify that to isolate x, we must remove A from the left side.
  2. Multiply both sides by A⁻¹ on the left: A⁻¹(Ax) = A⁻¹b.
  3. Simplify A⁻¹A to I, resulting in x = A⁻¹b.
  4. Verify the inverse by computing AA⁻¹; the result must be the identity matrix I.

Answer The solution is x = A⁻¹b, verified by ensuring AA⁻¹ = I.

Common mistakes

  • Assuming that a matrix inverse always exists or produces a perfectly accurate solution.
  • Confusing the scalar division operation with matrix multiplication, noting that order matters (left-multiplication).
  • Believing that the pseudo-inverse is only for non-square matrices, whereas it is also used for poorly behaved square matrices.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

This is a text lesson: it has no animation code yet.

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.