Content
Summary
Regression to the mean is a statistical phenomenon where extreme initial measurements tend to be less extreme in subsequent paired observations, a concept introduced by Francis Galton in 1886. This effect arises because measurements consist of an intrinsic property and random noise, causing extreme values to regress toward the average when noise is not perfectly correlated. Mathematically, for standardized variables, the predicted value is the product of the correlation coefficient and the observed value, shrinking the estimate toward the mean.
Intuition
Imagine measuring a person's height or performance twice. The first measurement is not just their true ability but also includes a bit of random luck or error (noise). If the first measurement is extremely high, it likely involved some favorable noise. Since noise is random, the second measurement is unlikely to have the same favorable noise, so the value will likely be closer to the average. This is not because the person changed, but because the random component fluctuates. If the correlation between the two measurements were perfect (1), there would be no noise, and the second measurement would be identical to the first. However, since correlation is usually less than 1, the second measurement shrinks toward the mean.
Key equations
The predicted value of the outcome Y given the predictor X, where both are standardized (mean 0, variance 1) and is the correlation coefficient.
The slope of the regression line when predicting X from Y (standardized variables).
The slope of the regression line when predicting Y from X (standardized variables), highlighting asymmetry.
Derivation
- Measurements consist of an intrinsic property and random noise.
- Extreme values often result from favorable random noise coinciding with the intrinsic value.
- Since noise is random, subsequent measurements exhibit less favorable noise, pulling values closer to the mean.
- For standardized variables, the predicted value is the product of the correlation coefficient and the observed value: .
- Since , multiplying by shrinks the predicted value toward zero (the mean).
- If correlation is 1, there is no noise, and regression to the mean disappears.
Worked example
In Galton's father-son height data, if the father's height is standardized to 2 standard deviations above the mean, and the correlation coefficient between father and son heights is 0.5, what is the predicted standardized height of the son?
- Identify the standardized father's height: X = 2.
- Identify the correlation coefficient: .
- Apply the regression equation for standardized variables: .
- Calculate: .
- The predicted son's height is 1 standard deviation above the mean, which is closer to the mean (0) than the father's height (2).
Answer 1 standard deviation above the mean.
Common mistakes
- Attributing regression to the mean to a causal intervention or intrinsic change rather than recognizing it as a statistical artifact of random variation.
- Assuming that extreme performance will always improve or worsen due to intervention, ignoring the role of random noise.
- Believing that regression to the mean implies a permanent shift in the underlying characteristic.