Biostatistics · Intermediate

Bootstrap resampling

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Bootstrap resampling estimates the uncertainty of a statistic by repeatedly drawing samples with replacement from the observed data. Each resample yields a new value of the statistic, and the spread of these values approximates its sampling distribution. It gives standard errors and confidence intervals without relying on closed-form formulas.

Intuition

We would love to repeat our study many times and see how much the result varies, but we only have one sample. The bootstrap treats the sample as a stand-in for the population: if the sample is representative, drawing from it with replacement mimics drawing new samples from the population. Each resample has the same size as the original, but some patients appear twice or more and others not at all. Computing the statistic (mean, median, correlation, AUC) on each resample shows how much it wobbles due to sampling alone. The histogram of these values is the bootstrap distribution, and its width estimates the standard error.

Key equations

θ^b∗=T(xb1∗,…,xbn∗),b=1,…,B
(1)

Statistic T computed on the b-th resample drawn with replacement from the original data of size n.

SE^boot=1B−1∑b=1B(θ^b∗−θ‾∗)2
(2)

Bootstrap standard error is the standard deviation of the resampled statistics.

[θ^(α/2)∗,θ^(1−α/2)∗]
(3)

Percentile confidence interval: the α/2 and 1−α/2 quantiles of the bootstrap distribution.

P(observation never chosen)=(1−1n)n≈e−1≈0.368
(4)

About 37% of the original observations are absent from any given resample.

Derivation

  1. Observed data x1..xn come from unknown distribution F; the empirical distribution F-hat puts mass 1/n on each observation.
  2. The sampling distribution of T under F is unknown, so we approximate it by the distribution of T under F-hat.
  3. Sampling from F-hat is the same as choosing n observations from the data with replacement, each with probability 1/n.
  4. Repeat B times (e.g., 2000) to get values of the statistic.
  5. The variability of these values estimates the true sampling variability; take their standard deviation as SE, or their quantiles as a confidence interval.
  6. The probability a given observation is missed in n draws is (1-1/n)^n, which tends to e^-1, explaining the ~63% unique observations per resample.

Worked example

Problem

Ten patients have reductions in systolic blood pressure (mmHg): 4, 7, 9, 3, 12, 6, 8, 5, 10, 6. Describe how to get a bootstrap 95% CI for the mean, and interpret a result where B=1000 resample means have 2.5th and 97.5th percentiles of 5.3 and 8.1.

  1. Original mean = (4+7+9+3+12+6+8+5+10+6)/10 = 70/10 = 7.0 mmHg.
  2. Draw 10 values with replacement from the data, compute the mean; repeat 1000 times.
  3. Sort the 1000 means; take the 25th and 975th values as the 2.5% and 97.5% percentiles.
  4. These are 5.3 and 8.1, so the interval is (5.3, 8.1).
  5. Approximate SE from the width: (8.1−5.3)/(2×1.96) ≈ 0.71 mmHg.

Answer Mean 7.0 mmHg with bootstrap 95% CI (5.3, 8.1) mmHg; SE about 0.71.

Common mistakes

  • Resampling without replacement, which just reproduces the original sample.
  • Using resamples of a different size than n, which changes the variability.
  • Using too few resamples (e.g., 20); use at least 1000 for intervals.
  • Resampling individual observations when data are clustered or repeated-measures; resample whole patients instead.
  • Believing the bootstrap fixes a small or biased sample; it only reflects the data you have.
  • Applying it to extreme statistics such as the maximum, where it fails.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

scene.py(176 lines)
import sys
import numpy as np
from manim import *
from blocks import *


def bootstrap_resample(data, n_bootstrap=500, stat_func=np.mean):
    """Perform bootstrap resampling and return statistics.
    
    Args:
        data: Original sample data (numpy array)
        n_bootstrap: Number of bootstrap samples
        stat_func: Function to compute on each bootstrap sample
        
    Returns:
        bootstrap_stats: Array of computed statistics
        ci_lower: Lower bound of 95% confidence interval
        ci_upper: Upper bound of 95% confidence interval
        original_stat: Statistic computed on original data
    """
    np.random.seed(42)
    n = len(data)
    bootstrap_stats = []
    
    for _ in range(n_bootstrap):
        # Resample with replacement
        sample = np.random.choice(data, size=n, replace=True)
        bootstrap_stats.append(stat_func(sample))
    
    bootstrap_stats = np.array(bootstrap_stats)
    alpha = 0.05
    ci_lower = np.percentile(bootstrap_stats, 100 * alpha / 2)
    ci_upper = np.percentile(bootstrap_stats, 100 * (1 - alpha / 2))
    original_stat = stat_func(data)
    
    return bootstrap_stats, ci_lower, ci_upper, original_stat


# Generate original sample data (simulating patient recovery times in days)
np.random.seed(42)
original_data = np.random.normal(loc=14.0, scale=3.0, size=30)
original_data = np.clip(original_data, 5, 25)  # Clip to realistic range

# Perform bootstrap resampling
bootstrap_stats, ci_lower, ci_upper, original_stat = bootstrap_resample(
    original_data, 
    n_bootstrap=1000, 
    stat_func=np.mean
)

# Compute histogram data for visualization
hist_bins = np.linspace(np.min(bootstrap_stats), np.max(bootstrap_stats), 50)
hist_counts, bin_edges = np.histogram(bootstrap_stats, bins=hist_bins)
hist_density = hist_counts / (np.sum(hist_counts) * (bin_edges[1] - bin_edges[0]))


class Main(Scene):
    def construct(self):
        self.camera.background_color = "#111111"
        
        TitleCard(self, "Bootstrap Resampling", "Estimating Confidence Intervals")
        
        # Create axes for the bootstrap distribution
        x_min = ci_lower - 2
        x_max = ci_upper + 2
        y_max = np.max(hist_density) * 1.2
        
        axes = Axes(
            x_range=[x_min, x_max, 1],
            y_range=[0, y_max, 0.05],
            x_length=10,
            y_length=5,
            axis_config={"include_tip": True, "color": C_DIM}
        )
        axes.to_edge(LEFT, buff=0.5)
        axes.to_edge(UP, buff=0.5)
        
        x_label = Text("Mean Recovery Time (days)", font_size=22, color=WHITE)
        x_label.next_to(axes.x_axis.get_end(), RIGHT, buff=0.2)
        y_label = Text("Density", font_size=22, color=WHITE)
        y_label.next_to(axes.y_axis.get_end(), UP, buff=0.2)
        
        self.play(Create(axes), FadeIn(x_label), FadeIn(y_label))
        
        # Plot original statistic
        orig_line = DashedLine(
            axes.c2p(original_stat, 0),
            axes.c2p(original_stat, y_max),
            color=C_ACCENT,
            dash_length=0.2
        )
        orig_label = Text(f"Original: {original_stat:.1f}", font_size=20, color=C_ACCENT)
        orig_label.next_to(axes.c2p(original_stat, y_max * 0.9), RIGHT, buff=0.2)
        
        self.play(Create(orig_line), FadeIn(orig_label))
        
        # Plot bootstrap distribution histogram
        # Create bars for the histogram
        bar_width = (bin_edges[1] - bin_edges[0]) * 0.9
        bars = VGroup()
        
        for i in range(len(hist_counts)):
            x = bin_edges[i]
            height = hist_density[i]
            bar = Rectangle(
                width=bar_width,
                height=height,
                fill_color=C_MAIN,
                fill_opacity=0.7,
                stroke_color=WHITE,
                stroke_width=1
            )
            bar.move_to(axes.c2p(x + (bin_edges[1] - bin_edges[0]) / 2, height / 2))
            bars.add(bar)
        
        self.play(Create(bars), run_time=2)
        
        # Highlight confidence interval region
        ci_lower_idx = np.searchsorted(hist_bins, ci_lower)
        ci_upper_idx = np.searchsorted(hist_bins, ci_upper)
        
        ci_bars = bars[ci_lower_idx:ci_upper_idx]
        ci_highlight = ci_bars.copy().set_fill(C_GOOD, opacity=0.9)
        
        self.play(
            FadeIn(ci_highlight),
            run_time=1.5
        )
        
        # Add CI labels
        ci_lower_label = Text(f"{ci_lower:.1f}", font_size=20, color=C_GOOD)
        ci_lower_label.next_to(axes.c2p(ci_lower, 0), DOWN, buff=0.2)
        
        ci_upper_label = Text(f"{ci_upper:.1f}", font_size=20, color=C_GOOD)
        ci_upper_label.next_to(axes.c2p(ci_upper, 0), DOWN, buff=0.2)
        
        # Use a simpler bracket approach without Bracket class
        ci_bracket = VGroup(
            Line(axes.c2p(ci_lower, 0.2), axes.c2p(ci_lower, 0.5), color=C_GOOD),
            Line(axes.c2p(ci_upper, 0.2), axes.c2p(ci_upper, 0.5), color=C_GOOD),
            Line(axes.c2p(ci_lower, 0.5), axes.c2p(ci_upper, 0.5), color=C_GOOD)
        )
        ci_text = Text("95% CI", font_size=22, color=C_GOOD)
        ci_text.next_to(Line(axes.c2p(ci_lower, 0.5), axes.c2p(ci_upper, 0.5)), UP, buff=0.1)
        
        self.play(FadeIn(ci_lower_label), FadeIn(ci_upper_label))
        self.play(Create(ci_bracket), FadeIn(ci_text))
        
        # Show key insight
        StepCaption(self, "Bootstrap: Resample with replacement to estimate uncertainty")
        
        self.wait(2)


if __name__ == "__main__" and "--selftest" in sys.argv:
    # Test that bootstrap resampling produces valid results
    assert len(bootstrap_stats) == 1000, "Should have 1000 bootstrap samples"
    
    # Test that confidence interval bounds are valid
    assert ci_lower < ci_upper, "CI lower bound should be less than upper bound"
    
    # Test that original statistic is within the confidence interval (usually)
    # Note: This is probabilistic, so we check it's close to the center
    ci_center = (ci_lower + ci_upper) / 2
    assert abs(original_stat - ci_center) < 1.0, f"Original stat {original_stat} should be near CI center {ci_center}"
    
    # Test that bootstrap stats are reasonable (mean of bootstrap stats should be close to original)
    bootstrap_mean = np.mean(bootstrap_stats)
    assert abs(bootstrap_mean - original_stat) < 0.5, f"Bootstrap mean {bootstrap_mean} should be close to original {original_stat}"
    
    # Test that the standard error is reasonable
    bootstrap_se = np.std(bootstrap_stats)
    original_se = np.std(original_data) / np.sqrt(len(original_data))
    assert abs(bootstrap_se - original_se) < 0.5, f"Bootstrap SE {bootstrap_se} should be close to original SE {original_se}"
    
    print("ok")

The Manim scene code that draws this lesson's animation in the Signals AI app (Python, Manim Community).

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.