Biostatistics · Intermediate

Bayesian prior to posterior

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Bayesian updating combines what we believed before seeing data (the prior) with how well each hypothesis explains the data (the likelihood) to give an updated belief (the posterior). In medicine this is how a screening test result changes the probability that a patient has a disease.

Intuition

Imagine a bar of probability split among competing hypotheses, such as 'diseased' and 'healthy'. The prior sets how wide each slice is before testing. When a test result arrives, each slice is stretched or shrunk according to how likely that result would be under that hypothesis. A hypothesis that predicted the data well grows, and one that predicted it poorly shrinks. Finally the slices are rescaled so they again total one. That rescaled result is the posterior. A rare disease starts with a tiny slice, so even a fairly accurate positive test may not make it the majority, which is why base rates matter.

Key equations

P(H∣D)=P(D∣H)P(H)P(D)
(1)

Bayes' theorem: posterior equals likelihood times prior divided by the evidence.

P(D)=∑iP(D∣Hi)P(Hi)
(2)

The evidence is the total probability of the data across all hypotheses; it normalises the posterior.

Posterior∝Likelihood×Prior
(3)

The shape of the update before normalising.

P(H∣D)P(¬H∣D)=P(D∣H)P(D∣¬H)⋅P(H)P(¬H)
(4)

Odds form: posterior odds equal likelihood ratio times prior odds.

Derivation

  1. Start from the definition of conditional probability: P(H and D) = P(H|D)P(D).
  2. Also P(H and D) = P(D|H)P(H), since the joint probability can be factored either way.
  3. Set the two expressions equal: P(H|D)P(D) = P(D|H)P(H).
  4. Divide by P(D), assuming P(D) > 0, to get P(H|D) = P(D|H)P(H)/P(D).
  5. Expand P(D) with the law of total probability over the exhaustive, exclusive hypotheses: P(D) = sum of P(D|Hi)P(Hi).
  6. Because P(D) does not depend on H, the posterior is proportional to likelihood times prior; dividing by P(D) just makes the probabilities sum to one.

Worked example

Problem

A disease has prevalence 1%. A test has sensitivity 90% and specificity 95%. A patient tests positive. What is the probability they have the disease?

  1. Prior: P(D)=0.01, so P(no D)=0.99.
  2. Likelihoods: P(+|D)=0.90; P(+|no D)=1-0.95=0.05.
  3. Evidence: P(+)=0.90*0.01+0.05*0.99=0.009+0.0495=0.0585.
  4. Posterior: P(D|+)=0.009/0.0585≈0.154.
  5. Check with odds: prior odds 0.01/0.99≈0.0101; likelihood ratio 0.90/0.05=18; posterior odds≈0.182, which gives probability 0.182/1.182≈0.154.

Answer About 15%. Despite a positive test, most positives are false because the disease is rare.

Common mistakes

  • Confusing P(positive|disease) with P(disease|positive), the transposed conditional fallacy.
  • Ignoring the base rate (prior prevalence).
  • Forgetting to normalise by the evidence so posteriors do not sum to one.
  • Using specificity where the false positive rate (1 - specificity) is needed.
  • Treating a prior of exactly 0 or 1 as updatable; no data can change it.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

scene.py(134 lines)
import sys
import numpy as np
from manim import *
from blocks import *

# Parameters
N_TRIALS = 100
N_SUCCESS = 65
PRIOR_ALPHA = 2.0
PRIOR_BETA = 2.0

def beta_pdf(x, alpha, beta):
    """Compute Beta PDF values using scipy if available, else fallback."""
    try:
        from scipy.special import beta as beta_func
        if x <= 0 or x >= 1:
            return 0.0
        return (x**(alpha-1)) * ((1-x)**(beta-1)) / beta_func(alpha, beta)
    except ImportError:
        # Fallback approximation if scipy is missing (though usually present in env)
        # Using log-gamma for stability if needed, but simple form is fine for visualization
        if x <= 0 or x >= 1:
            return 0.0
        # Approximate beta function with gamma for simple cases or just return 0 to avoid crash
        # For this specific animation, we assume scipy is available for correctness.
        return 0.0

class Main(Scene):
    def construct(self):
        # Title
        TitleCard(self, "Bayesian Update", "Prior to Posterior with Binomial Data")

        # Setup Axes for Beta Distribution
        # Using explicit Axes to control label placement precisely and avoid cutoff
        axes = Axes(
            x_range=[0, 1, 0.2],
            y_range=[0, 15, 5],
            x_length=8.0,
            y_length=4.0,
            axis_config={"color": WHITE, "include_numbers": False}
        )
        axes.to_edge(LEFT, buff=0.5)
        axes.to_edge(DOWN, buff=0.5)
        
        # Labels
        x_label = Text("Probability of Success ($\\theta$)", font_size=24, color=WHITE)
        x_label.next_to(axes.x_axis, DOWN, buff=0.2)
        
        y_label = Text("Density", font_size=24, color=WHITE)
        y_label.next_to(axes.y_axis, LEFT, buff=0.2)
        
        self.play(Create(axes), Write(x_label), Write(y_label), run_time=1.5)

        # Plot Prior
        prior_curve = axes.plot(
            lambda x: beta_pdf(x, PRIOR_ALPHA, PRIOR_BETA),
            x_range=[0.01, 0.99],
            color=C_MAIN,
            use_smoothing=True
        )
        self.play(Create(prior_curve), run_time=2)
        
        # Label Prior
        prior_label = Text("Prior", font_size=28, color=C_MAIN)
        # Place label near the peak of the prior (which is at 0.5)
        prior_peak_x = 0.5
        prior_peak_y = beta_pdf(prior_peak_x, PRIOR_ALPHA, PRIOR_BETA)
        prior_label.next_to(axes.c2p(prior_peak_x, prior_peak_y), UP, buff=0.2)
        self.play(FadeIn(prior_label), run_time=0.8)

        # Compute posterior parameters
        post_alpha = PRIOR_ALPHA + N_SUCCESS
        post_beta = PRIOR_BETA + (N_TRIALS - N_SUCCESS)
        
        # Compute posterior values for smooth transition
        x_vals = np.linspace(0.01, 0.99, 100)
        y_post = np.array([beta_pdf(x, post_alpha, post_beta) for x in x_vals])
        
        # Create posterior curve
        posterior_curve = axes.plot(
            lambda x: beta_pdf(x, post_alpha, post_beta),
            x_range=[0.01, 0.99],
            color=C_ACCENT,
            use_smoothing=True
        )
        
        # Animate transition: Fade out prior, Fade in posterior
        self.play(FadeOut(prior_curve), FadeIn(posterior_curve), run_time=2)
        self.play(FadeOut(prior_label), run_time=0.5)
        
        # Label Posterior
        post_label = Text("Posterior", font_size=28, color=C_ACCENT)
        peak_idx = np.argmax(y_post)
        peak_x = x_vals[peak_idx]
        peak_y = y_post[peak_idx]
        post_label.next_to(axes.c2p(peak_x, peak_y), UP, buff=0.2)
        self.play(FadeIn(post_label), run_time=0.8)

        # Show Data Impact
        data_text = Text(f"Data: {N_SUCCESS}/{N_TRIALS}", font_size=24, color=WHITE)
        data_text.to_edge(DOWN, buff=0.5)
        self.play(Write(data_text), run_time=1)

        # Explanation
        eq = MathTex(r"P(\theta \mid D) \propto P(D \mid \theta) \cdot P(\theta)", font_size=30, color=WHITE)
        eq.to_edge(UP, buff=0.8)
        self.play(Write(eq), run_time=1.5)

        StepCaption(self, "The posterior combines prior belief with observed data likelihood.")
        self.wait(2)


if __name__ == "__main__" and "--selftest" in sys.argv:
    from scipy.integrate import quad
    from scipy.special import beta as beta_func
    
    # Check prior integral
    integral_prior, _ = quad(lambda x: beta_pdf(x, PRIOR_ALPHA, PRIOR_BETA), 0, 1)
    assert abs(integral_prior - 1.0) < 1e-6
    
    # Check posterior integral
    post_alpha = PRIOR_ALPHA + N_SUCCESS
    post_beta = PRIOR_BETA + (N_TRIALS - N_SUCCESS)
    integral_post, _ = quad(lambda x: beta_pdf(x, post_alpha, post_beta), 0, 1)
    assert abs(integral_post - 1.0) < 1e-6
    
    # Check peak location
    mode_prior = (PRIOR_ALPHA - 1) / (PRIOR_ALPHA + PRIOR_BETA - 2)
    mode_post = (post_alpha - 1) / (post_alpha + post_beta - 2)
    
    assert abs(mode_prior - 0.5) < 1e-6
    assert abs(mode_post - 0.6470588235294118) < 1e-6
    
    print("ok")

The Manim scene code that draws this lesson's animation in the Signals AI app (Python, Manim Community).

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.