Content
Summary
Bayesian updating combines what we believed before seeing data (the prior) with how well each hypothesis explains the data (the likelihood) to give an updated belief (the posterior). In medicine this is how a screening test result changes the probability that a patient has a disease.
Intuition
Imagine a bar of probability split among competing hypotheses, such as 'diseased' and 'healthy'. The prior sets how wide each slice is before testing. When a test result arrives, each slice is stretched or shrunk according to how likely that result would be under that hypothesis. A hypothesis that predicted the data well grows, and one that predicted it poorly shrinks. Finally the slices are rescaled so they again total one. That rescaled result is the posterior. A rare disease starts with a tiny slice, so even a fairly accurate positive test may not make it the majority, which is why base rates matter.
Key equations
Bayes' theorem: posterior equals likelihood times prior divided by the evidence.
The evidence is the total probability of the data across all hypotheses; it normalises the posterior.
The shape of the update before normalising.
Odds form: posterior odds equal likelihood ratio times prior odds.
Derivation
- Start from the definition of conditional probability: P(H and D) = P(H|D)P(D).
- Also P(H and D) = P(D|H)P(H), since the joint probability can be factored either way.
- Set the two expressions equal: P(H|D)P(D) = P(D|H)P(H).
- Divide by P(D), assuming P(D) > 0, to get P(H|D) = P(D|H)P(H)/P(D).
- Expand P(D) with the law of total probability over the exhaustive, exclusive hypotheses: P(D) = sum of P(D|Hi)P(Hi).
- Because P(D) does not depend on H, the posterior is proportional to likelihood times prior; dividing by P(D) just makes the probabilities sum to one.
Worked example
A disease has prevalence 1%. A test has sensitivity 90% and specificity 95%. A patient tests positive. What is the probability they have the disease?
- Prior: P(D)=0.01, so P(no D)=0.99.
- Likelihoods: P(+|D)=0.90; P(+|no D)=1-0.95=0.05.
- Evidence: P(+)=0.90*0.01+0.05*0.99=0.009+0.0495=0.0585.
- Posterior: P(D|+)=0.009/0.0585≈0.154.
- Check with odds: prior odds 0.01/0.99≈0.0101; likelihood ratio 0.90/0.05=18; posterior odds≈0.182, which gives probability 0.182/1.182≈0.154.
Answer About 15%. Despite a positive test, most positives are false because the disease is rare.
Common mistakes
- Confusing P(positive|disease) with P(disease|positive), the transposed conditional fallacy.
- Ignoring the base rate (prior prevalence).
- Forgetting to normalise by the evidence so posteriors do not sum to one.
- Using specificity where the false positive rate (1 - specificity) is needed.
- Treating a prior of exactly 0 or 1 as updatable; no data can change it.