Content
Summary
Probability theory provides the essential framework for distinguishing statistical significance from random chance in research. Bayesian inference updates prior beliefs with empirical data to calculate posterior probabilities, while ROC curves and the Area Under the Curve (AUC) evaluate diagnostic test accuracy by plotting sensitivity against one minus specificity.
Intuition
Think of probability as the lens that helps us decide if a pattern in data is real or just noise. In Bayesian thinking, we start with an initial guess (prior) based on general knowledge, such as knowing that 20% of adults have hypertension. When we get specific evidence, like a positive blood pressure reading, we update that guess to a more precise estimate (posterior). For diagnostic tests that don't give simple yes/no answers, we use ROC curves to see how well a test works at different cutoff points. The AUC summarizes this performance, representing the chance that a sick person scores higher than a healthy one.
Derivation
- Probability serves as the theoretical foundation to determine if observed differences are statistically significant or due to random chance.
- Prior probability is defined as the observer's best estimate of an event's likelihood before new empirical data is collected.
- Posterior probability is calculated by integrating the prior probability with new empirical data.
- As the volume of empirical data increases, the influence of the subjective prior probability on the final posterior inference diminishes.
- For tests with continuous variables, an ROC curve is constructed by plotting sensitivity on the y-axis against one minus specificity on the x-axis.
- The Area Under the Curve (AUC) is calculated using the trapezoidal rule and represents the probability that a randomly selected abnormal subject has a higher test rating than a randomly selected normal subject.
Worked example
In a radiology study, varying the CT scan cutoff from 4+ to 5+ changes sensitivity from 0.86 to 0.65 and specificity from 0.78 to 0.97. If the computed AUC is 0.89, what does this value signify regarding the radiologist's ability to differentiate patients?
- Identify that the AUC is a summary measure of overall diagnostic accuracy.
- Recall that the AUC equals the probability that a randomly selected abnormal subject will have a higher test rating than a randomly selected normal subject.
- Interpret the value 0.89 as an 89% likelihood.
- Conclude that there is an 89% probability the radiologist will accurately differentiate between healthy and diseased patients based on the CT scan scores.
Answer The AUC of 0.89 signifies an 89% likelihood that the radiologist will accurately differentiate between healthy and diseased patients by relying on the sequence of their CT scan scores.
Common mistakes
- Assuming that a higher AUC always makes a test superior, ignoring cases where specific sensitivity or specificity thresholds are clinically critical.
- Confusing prior probability with posterior probability; the prior is the estimate before data, while the posterior is after integrating data.
- Believing that subjective priors always lead to incorrect results; sufficient empirical data diminishes the impact of the prior.
- Thinking that ROC curves are only for binary outcomes; they are specifically used when tests yield continuous variables or multiple categories.