AI, machine learning and deep learning · Introductory

A neuron and a forward pass

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

An artificial neuron takes several numerical inputs, multiplies each by a weight, adds a bias, and passes the result through a nonlinear activation function. A forward pass is simply this computation carried out from inputs to output. Geometrically, a neuron splits input space with a line (or plane) and the activation shapes the response.

Intuition

Think of a neuron as a tiny voting machine. Each input casts a vote, and its weight says how much that input matters and in which direction. The bias is a built-in tendency to fire or stay quiet. The votes are summed into one number, z. Geometrically, z is a signed distance-like measure from a line in input space: points on one side give positive z, points on the other give negative z. The activation function, such as a sigmoid, then squashes z into a useful range like 0 to 1, so the neuron reports how confidently a point lies on the positive side. Stacking neurons into layers and feeding outputs forward gives a network, and the forward pass is just repeating this recipe layer by layer.

Key equations

z=w1x1+w2x2+b=𝐰⋅𝐱+b
(1)

Weighted sum of inputs plus bias (pre-activation).

a=σ(z)=11+e−z
(2)

Sigmoid activation squashes z into the range 0 to 1.

w1x1+w2x2+b=0
(3)

The decision boundary: a line in the (x1, x2) plane where z = 0.

𝐚(l)=σ(W(l)𝐚(l−1)+𝐛(l))
(4)

Forward pass for a whole layer using a weight matrix.

Derivation

  1. Start with inputs x1 and x2 arranged as a point in the plane.
  2. Give each input a weight: the weight vector w = (w1, w2) points in a direction in this plane.
  3. The dot product w·x measures how far x extends along the direction of w, scaled by its length.
  4. Adding the bias b shifts this value, so the zero level moves away from the origin.
  5. Setting z = 0 gives the line w1 x1 + w2 x2 + b = 0; w is perpendicular to this line.
  6. Points with z > 0 lie on the side w points toward; z < 0 on the other side.
  7. Applying sigmoid turns z into a = σ(z); a = 0.5 exactly on the line and approaches 1 or 0 far from it.
  8. For many neurons, stack weight vectors as rows of W to compute all z values at once, then repeat for the next layer.

Worked example

Problem

A neuron has weights w = (2, -1), bias b = -1, and sigmoid activation. Compute its output for input x = (3, 2). Which side of the boundary is the point on?

  1. Compute the weighted sum: 2·3 + (-1)·2 = 6 - 2 = 4.
  2. Add the bias: z = 4 - 1 = 3.
  3. Apply sigmoid: a = 1/(1+e^{-3}).
  4. e^{-3} ≈ 0.0498, so a ≈ 1/1.0498 ≈ 0.953.
  5. Since z = 3 > 0, the point lies on the positive side of the line 2x1 - x2 - 1 = 0.

Answer z = 3, output a ≈ 0.95, on the positive side of the boundary.

Common mistakes

  • Forgetting the bias, which forces the boundary through the origin.
  • Thinking the weights are the inputs or that they change during a forward pass; they are fixed while computing.
  • Confusing z (pre-activation) with a (post-activation).
  • Believing a single neuron can separate any data; it can only draw one straight boundary.
  • Omitting the nonlinearity: stacked linear layers collapse into a single linear map.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

scene.py(81 lines)
import sys
import numpy as np
from manim import *
from blocks import *

# Real computation: a single neuron with fixed inputs, weights, bias, sigmoid
X = [1.0, 0.5, -1.0]
W = [0.8, -0.4, 0.3]
B = 0.1


def sigmoid(z):
    """Logistic activation."""
    return 1.0 / (1.0 + np.exp(-z))


def forward(x, w, b):
    """Forward pass of one neuron; records the states."""
    terms = [wi * xi for wi, xi in zip(w, x)]
    z = sum(terms) + b
    a = sigmoid(z)
    return {"terms": terms, "z": z, "a": a}


TRACE = forward(X, W, B)


class Main(Scene):
    def construct(self):
        # Title
        TitleCard(self, "A neuron and a forward pass")

        # Diagram: three inputs, one neuron
        ins = VGroup(*[Circle(radius=0.3, color=C_MAIN, fill_color=BLACK, fill_opacity=1)
                       for _ in X]).arrange(DOWN, buff=0.8).move_to(LEFT * 4 + DOWN * 0.2)
        neuron = Circle(radius=0.5, color=C_ACCENT, fill_color=BLACK, fill_opacity=1).move_to(RIGHT * 1)
        out = Text("a", font_size=28, color=C_GOOD).move_to(RIGHT * 4.5 + DOWN * 0.2)
        neuron.match_y(out)
        in_labels = VGroup(*[Text(f"x={x}", font_size=24).next_to(c, LEFT, buff=0.15)
                             for x, c in zip(X, ins)])
        lines = VGroup()
        w_labels = VGroup()
        for c, w in zip(ins, W):
            ln = Line(c.get_right(), neuron.get_left(), color=C_DIM)
            lines.add(ln)
            t = Text(f"w={w}", font_size=24, color=C_ACCENT)
            t.move_to(ln.point_from_proportion(0.45) + 0.28 * UP)
            w_labels.add(t)
        arrow = Arrow(neuron.get_right(), out.get_left(), buff=0.1, color=C_DIM)
        self.play(Create(ins), FadeIn(in_labels), Create(neuron))
        self.play(Create(lines), FadeIn(w_labels), Create(arrow), FadeIn(out))
        StepCaption(self, "Each input is multiplied by its weight")

        # Show products one by one (recorded terms)
        prods = VGroup()
        for k, t in enumerate(TRACE["terms"]):
            p = Text(f"{W[k]}*{X[k]} = {t:.2f}", font_size=24, color=C_MAIN)
            prods.add(p)
        prods.arrange(DOWN, aligned_edge=LEFT, buff=0.2).to_corner(UR, buff=0.5).shift(DOWN * 0.8)
        for k in range(3):
            self.play(lines[k].animate.set_color(C_ACCENT).set_stroke(width=6), FadeIn(prods[k]), run_time=0.7)

        # Sum with bias
        StepCaption(self, "Add the bias, then apply the sigmoid")
        zt = Text(f"z = sum + b = {TRACE['z']:.2f}", font_size=26, color=C_ACCENT)
        zt.next_to(prods, DOWN, buff=0.3, aligned_edge=LEFT)
        self.play(Write(zt))
        at = Text(f"a = sigmoid(z) = {TRACE['a']:.3f}", font_size=26, color=C_GOOD)
        at.next_to(zt, DOWN, buff=0.2, aligned_edge=LEFT)
        self.play(Write(at), neuron.animate.set_fill(C_ACCENT, opacity=0.1 + 0.9 * TRACE["a"]))
        self.wait(0.8)
        StepCaption(self, "Forward pass: weighted sum, plus bias, through an activation")
        self.wait(1)


if __name__ == "__main__" and "--selftest" in sys.argv:
    assert abs(TRACE["z"] - (0.8 - 0.2 - 0.3 + 0.1)) < 1e-9
    assert abs(TRACE["a"] - 1 / (1 + np.exp(-0.4))) < 1e-9
    assert 0 < TRACE["a"] < 1
    assert abs(sigmoid(0.0) - 0.5) < 1e-12
    print("ok")

The Manim scene code that draws this lesson's animation in the Signals AI app (Python, Manim Community).

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.