AI, machine learning and deep learning Ā· Introductory

Perceptron learning rule

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

The perceptron is a single artificial neuron that classifies inputs into two classes using a weighted sum and a threshold. The perceptron learning rule nudges the weights whenever a training example is misclassified, moving the decision boundary toward the correct side. If the classes are linearly separable, this procedure is guaranteed to converge.

Intuition

Imagine a line separating two groups of points. The perceptron draws a line using its weights. When it labels a point wrongly, it tilts and shifts the line a little so that the point is more likely to fall on the correct side next time. If a positive point is called negative, we add a fraction of that point's input vector to the weights, making the weighted sum bigger for that input. If a negative point is called positive, we subtract. Correct points cause no change. Repeating this over the data, the line wiggles until no point is misclassified, provided a separating line exists.

Key equations

y^={1if š°ā‹…š±+b≄00otherwise
(1)

Prediction: threshold of the weighted sum of inputs plus bias.

š°ā†š°+Ī·(yāˆ’y^)š±
(2)

Weight update: change is proportional to the error and the input.

b←b+Ī·(yāˆ’y^)
(3)

Bias update uses the same error with a constant input of 1.

Derivation

  1. Define the net input z = wĀ·x + b and predict 1 if z ≄ 0, else 0.
  2. Error is e = y āˆ’ Å·, which is 0 if correct, +1 if a true 1 was predicted 0, and āˆ’1 if a true 0 was predicted 1.
  3. We want to change z in the direction that reduces the error: increase z when e = +1, decrease when e = āˆ’1.
  4. Updating w by Ī·eĀ·x changes z by Ī·e(xĀ·x) = Ī·e‖x‖², which has the sign of e, so z moves the right way.
  5. Apply the same logic to b, treating it as a weight on a constant input of 1.
  6. When e = 0 nothing changes, so updates occur only on mistakes.

Worked example

Problem

Start with w = (0, 0), b = 0, Ī· = 1. Sample A: x = (1, 2), y = 1. Sample B: x = (2, 1), y = 0. Present A, then B, then A again. Give the final weights and bias and whether A is then correct.

  1. Sample A: z = 0, so Å· = 1 (since z ≄ 0). Correct, e = 0, no change.
  2. Sample B: z = 0, so Å· = 1 but y = 0, e = āˆ’1.
  3. Update w = (0,0) + (āˆ’1)(2,1) = (āˆ’2, āˆ’1); b = 0 āˆ’ 1 = āˆ’1.
  4. Sample A again: z = (āˆ’2)(1) + (āˆ’1)(2) āˆ’ 1 = āˆ’5, so Å· = 0 but y = 1, e = +1.
  5. Update w = (āˆ’2,āˆ’1) + (1,2) = (āˆ’1, 1); b = āˆ’1 + 1 = 0.
  6. Check A: z = āˆ’1 + 2 + 0 = 1 ≄ 0, so Å· = 1, correct. Check B: z = āˆ’2 + 1 = āˆ’1, Å· = 0, correct.

Answer w = (āˆ’1, 1), b = 0; both samples are now classified correctly.

Common mistakes

  • Updating weights on correctly classified points.
  • Forgetting to update the bias along with the weights.
  • Using the wrong sign of the error, which pushes the boundary away from the point.
  • Expecting convergence on data that is not linearly separable, such as XOR.
  • Confusing the perceptron rule with gradient descent on a smooth loss; the step output is not differentiable.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

scene.py(180 lines)
import sys
import numpy as np
from manim import *
from blocks import *


def perceptron_trace(data, labels, lr=0.5, max_epochs=10):
    """
    Run perceptron learning on 2D data.
    Returns list of snapshots: {'w': [w0, w1, w2], 'epoch': int, 'error': int}.
    w = [bias, w1, w2] corresponding to [1, x, y].
    """
    X = np.array(data)
    Y = np.array(labels)
    n_samples = len(Y)
    
    # Initialize weights to zero
    w = np.array([0.0, 0.0, 0.0])
    
    trace = []
    
    for epoch in range(max_epochs):
        errors = 0
        for i in range(n_samples):
            x_vec = np.array([1.0, X[i, 0], X[i, 1]])
            pred = np.sign(np.dot(w, x_vec))
            # Handle 0 prediction as error (or negative class)
            if pred == 0:
                pred = -1 
            
            if pred != Y[i]:
                errors += 1
                w += lr * Y[i] * x_vec
        
        trace.append({
            'w': w.copy(),
            'epoch': epoch,
            'errors': errors
        })
        
        if errors == 0:
            break
            
    return trace


# Data: Linearly separable points
# Class 1: (1,1), (2,2), (1,2)
# Class -1: (-1,-1), (-2,-2), (-1,-2)
DATA = [
    [1, 1], [2, 2], [1, 2],
    [-1, -1], [-2, -2], [-1, -2]
]
LABELS = [1, 1, 1, -1, -1, -1]

TRACE = perceptron_trace(DATA, LABELS, lr=0.5, max_epochs=10)


class Main(Scene):
    def construct(self):
        self.camera.background_color = "#101418"
        
        # Title
        TitleCard(self, "Perceptron Learning", "Iterative weight updates")
        
        # Setup Plot
        plot = CurvePlot(
            lambda x: 0, # Placeholder, we draw line manually
            x_range=(-3, 3),
            y_range=(-3, 3),
            x_label="x",
            y_label="y",
            x_length=6,
            y_length=6
        )
        plot.shift(UP * 0.2)
        
        # Draw data points
        dot_group = VGroup()
        for i, pt in enumerate(DATA):
            x, y = pt
            color = GREEN if LABELS[i] == 1 else RED
            dot = Dot(point=[x, y, 0], color=color, radius=0.15)
            dot_group.add(dot)
            
        # Initial state
        self.play(Create(plot.axes), FadeIn(plot.labels))
        self.play(FadeIn(dot_group))
        
        info_box = VGroup()
        weight_text = Text("w = [0, 0, 0]", font_size=24, color=C_ACCENT)
        err_text = Text("Epoch 0: Errors = 6", font_size=24, color=WHITE)
        info_box.add(weight_text, err_text)
        info_box.arrange(DOWN, aligned_edge=LEFT, buff=0.2)
        info_box.to_corner(UL, buff=0.5)
        
        # Draw initial boundary (undefined, so just hide or draw a faint line)
        # Actually, let's just start animating from the first trace
        
        self.play(FadeIn(info_box))
        
        # Animate traces
        for k, snap in enumerate(TRACE):
            w = snap['w']
            epoch = snap['epoch']
            errors = snap['errors']
            
            # Update text
            w_str = f"[{w[0]:.1f}, {w[1]:.1f}, {w[2]:.1f}]"
            new_weight = Text(f"w = {w_str}", font_size=24, color=C_ACCENT)
            new_err = Text(f"Epoch {epoch}: Errors = {errors}", font_size=24, color=WHITE)
            
            # Rebuild info box content
            new_info = VGroup(new_weight, new_err)
            new_info.arrange(DOWN, aligned_edge=LEFT, buff=0.2)
            new_info.move_to(info_box.get_center())
            
            # Draw decision boundary
            # y = -(w0 + w1*x) / w2
            line = None
            if abs(w[2]) > 1e-6:
                def boundary(x_val):
                    return -(w[0] + w[1] * x_val) / w[2]
                
                # Clip to view
                x_vals = np.linspace(-3, 3, 100)
                y_vals = [boundary(x) for x in x_vals]
                
                # Create line from points
                line_points = np.column_stack([x_vals, y_vals, np.zeros_like(x_vals)])
                # Filter points outside view
                valid = (np.array(y_vals) >= -3) & (np.array(y_vals) <= 3)
                line_points = line_points[valid]
                
                if len(line_points) > 1:
                    line = Line(
                        line_points[0], 
                        line_points[-1], 
                        color=WHITE, 
                        stroke_width=3
                    )
                    # Draw line
                    self.play(Create(line), run_time=0.5)
                else:
                    line = None
            else:
                line = None
            
            # Update info
            self.play(
                Transform(weight_text, new_weight),
                Transform(err_text, new_err),
                run_time=0.6
            )
            
            if errors == 0:
                StepCaption(self, "Converged! Weights stable.")
                break
            else:
                StepCaption(self, f"Epoch {epoch}: {errors} errors remain, update weights.")

        StepCaption(self, "The line separates the classes correctly.")
        self.wait(1)


if __name__ == "__main__" and "--selftest" in sys.argv:
    # Check that the final weights correctly classify all points
    final_w = TRACE[-1]['w']
    for i, pt in enumerate(DATA):
        x_vec = np.array([1.0, pt[0], pt[1]])
        pred = np.sign(np.dot(final_w, x_vec))
        if pred == 0: pred = -1
        assert pred == LABELS[i], f"Failed at point {pt}: pred {pred} != {LABELS[i]}"
    
    # Check that errors decreased or stayed 0
    errors_list = [t['errors'] for t in TRACE]
    for i in range(len(errors_list) - 1):
        assert errors_list[i+1] <= errors_list[i], "Errors should not increase"
        
    print("ok")

The Manim scene code that draws this lesson's animation in the Signals AI app (Python, Manim Community).

ccBYNCĀ© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.