Content
Summary
The odds form of Bayes's theorem simplifies belief updating by expressing the relationship between prior and posterior probabilities as a ratio, effectively eliminating the normalizing constant p(D). It states that posterior odds are equal to prior odds multiplied by the likelihood ratio, making manual or mental calculations more convenient. This form is particularly useful for comparing two mutually exclusive and collectively exhaustive hypotheses.
Intuition
When comparing two competing hypotheses, A and B, we often care more about their relative support than their absolute probabilities. By taking the ratio of their posterior probabilities, the complex normalizing constant p(D) cancels out, leaving a simpler relationship. This allows us to update our beliefs by simply multiplying the initial odds (prior odds) by how much more likely the observed data is under one hypothesis compared to the other (the likelihood ratio). This approach streamlines the Bayesian update process, especially when working without computational tools.
Key equations
The odds form of Bayes's theorem, where posterior odds equal prior odds times the likelihood ratio.
Conversion from odds o to probability.
Conversion from odds represented as a numerator (yes) and denominator (no) to probability.
Derivation
- Start with the standard probability form of Bayes's theorem: p(H|D) = p(H) p(D|H) / p(D).
- Consider two hypotheses, A and B, and write the posterior probability for each: p(A|D) = p(A) p(D|A) / p(D) and p(B|D) = p(B) p(D|B) / p(D).
- Form the ratio of the posterior probabilities: p(A|D) / p(B|D) = [p(A) p(D|A) / p(D)] / [p(B) p(D|B) / p(D)].
- Observe that the normalizing constant p(D) appears in both the numerator and denominator of the ratio, so it cancels out.
- Rearrange the remaining terms to group prior probabilities and likelihoods: p(A|D) / p(B|D) = [p(A) / p(B)] * [p(D|A) / p(D|B)].
- Recognize that p(A) / p(B) is the definition of prior odds o(A) and p(A|D) / p(B|D) is the posterior odds o(A|D).
- Substitute the odds notation to obtain the final equation: o(A|D) = o(A) * [p(D|A) / p(D|B)].
Worked example
In the cookie problem, Bowl 1 contains 3 vanilla cookies and 1 chocolate cookie, while Bowl 2 contains 1 vanilla and 1 chocolate cookie. You randomly pick a bowl (prior probability 0.5 for each) and draw a vanilla cookie. What are the posterior odds that the cookie came from Bowl 1 versus Bowl 2?
- Identify hypotheses: A = Bowl 1, B = Bowl 2.
- Determine prior probabilities: p(A) = 0.5, p(B) = 0.5. Thus, prior odds o(A) = p(A)/p(B) = 1.
- Determine likelihoods: p(D|A) = 3/4 (probability of vanilla from Bowl 1), p(D|B) = 1/2 (probability of vanilla from Bowl 2).
- Calculate the likelihood ratio: p(D|A) / p(D|B) = (3/4) / (1/2) = 3/2 = 1.5.
- Apply the odds form equation: Posterior odds o(A|D) = Prior odds o(A) * Likelihood ratio = 1 * 1.5 = 1.5.
- Convert odds to probability if needed: Probability = 1.5 / (1.5 + 1) = 1.5 / 2.5 = 0.6.
Answer The posterior odds are 1.5 (or 3:2) in favor of Bowl 1. The probability is 0.6.
Common mistakes
- Confusing the probability form with the odds form; the odds form specifically uses ratios and omits the normalizing constant.
- Misinterpreting odds as probabilities; for example, odds of 5:1 against do not mean a probability of 5/6, but rather 1/6.
- Failing to recognize that the likelihood ratio is the ratio of p(D|A) to p(D|B), not the individual likelihoods alone.