Content
Summary
A ROC curve plots sensitivity against 1 minus specificity as the cutoff of a diagnostic test is varied. The area under the curve (AUC) summarizes how well the test separates diseased from healthy people: 0.5 is chance, 1.0 is perfect discrimination.
Intuition
Imagine test scores for healthy and diseased patients, two overlapping bell-shaped distributions. Any cutoff you choose labels everyone above it as positive. A low cutoff catches nearly all diseased patients but also flags many healthy ones; a high cutoff does the opposite. The ROC curve traces this trade-off by sweeping the cutoff from high to low, so the point moves from the bottom-left corner (0,0) to the top-right corner (1,1). The more the two distributions are separated, the more the curve bows toward the top-left corner. AUC has a neat probabilistic meaning: it is the chance that a randomly chosen diseased patient has a higher score than a randomly chosen healthy one.
Key equations
Proportion of diseased patients correctly called positive (y-axis).
Proportion of healthy patients wrongly called positive (x-axis).
Area under the ROC curve.
Probability that a random diseased patient scores higher than a random healthy one, ties counted half.
Youden index, used to choose an optimal cutoff.
Derivation
- Choose a cutoff c; classify a patient as positive if score S ≥ c.
- For that c, count TP, FN, FP, TN in a 2x2 table and compute TPR and FPR.
- Lower c step by step: TPR and FPR can only stay the same or increase, so the curve is monotone from (0,0) to (1,1).
- Plot each (FPR, TPR) pair and join them; this is the empirical ROC curve.
- Compute the area with the trapezoidal rule between successive points.
- Equivalently, comparing all diseased–healthy pairs and counting the fraction in which the diseased score is higher (ties = 0.5) gives the same number, the Mann–Whitney U statistic divided by n_D·n_H.
Worked example
A test gives these (FPR, TPR) points: (0,0), (0.2,0.6), (0.5,0.9), (1,1). Estimate the AUC and the Youden-optimal point.
- Trapezoid 1: width 0.2, mean height (0+0.6)/2=0.3, area 0.06.
- Trapezoid 2: width 0.3, mean height (0.6+0.9)/2=0.75, area 0.225.
- Trapezoid 3: width 0.5, mean height (0.9+1)/2=0.95, area 0.475.
- Sum: 0.06+0.225+0.475=0.76.
- Youden J = TPR − FPR: 0.4 at point 2, 0.4 at point 3 (0.9−0.5), so both tie.
Answer AUC ≈ 0.76 (fair discrimination); the points (0.2,0.6) and (0.5,0.9) tie with J = 0.4.
Common mistakes
- Thinking AUC = 0.5 means 50% accuracy in a particular sample; it means no discrimination.
- Ignoring prevalence: ROC and AUC do not depend on prevalence, but predictive values do.
- Choosing the cutoff closest to the corner automatically, ignoring the different clinical costs of false negatives and false positives.
- Plotting specificity instead of 1 − specificity on the x-axis.
- Treating AUC as calibration; a test can discriminate well yet give poorly calibrated probabilities.
- Comparing AUCs without confidence intervals or a formal test such as DeLong.