Content
Summary
Heterogeneity in meta-analysis refers to the variation in true effect sizes across studies, distinct from random within-study error. The Q statistic quantifies this by comparing observed dispersion to the dispersion expected from sampling error alone, isolating true between-studies variation.
Intuition
When combining studies, we observe a total amount of variation in their results. This total variation is a mix of two things: random noise within each study and real differences in the true effect sizes between studies. To find the true heterogeneity, we must subtract the expected random error from the total observed variation. The Q statistic helps us do this by looking at the ratio of observed variation to expected error. Because it is a ratio, it acts as a standardized measure, meaning the specific units or scale of the effect size do not change the comparison, allowing for consistent assessment across different types of studies.
Key equations
The primary definition of Q as the weighted sum of squares of deviations from the summary effect.
An alternative form highlighting Q as a standardized measure using standard error.
An equivalent computational formula for Q.
Derivation
- Partition the total observed variation into two components: variation due to true heterogeneity and variation due to within-study error.
- Compute the total amount of study-to-study variation actually observed.
- Estimate the expected variation under the assumption that all studies share the same true effect size.
- Identify the excess variation as the difference between observed and expected variation, assuming it reflects real differences.
- Calculate Q as the weighted sum of squares, which captures the ratio of observed variation to within-study error.
Worked example
A researcher calculates the weighted sum of squared deviations for a meta-analysis and finds a Q value of 15.2 with 4 degrees of freedom. How does this value help interpret heterogeneity?
- Recognize that Q measures the ratio of observed variation to expected within-study error.
- Note that Q is a standardized measure, unaffected by the metric of the effect size index.
- Compare the observed Q value to the expected variation under the null hypothesis of no heterogeneity.
- Conclude that a significant Q value indicates that the observed variation exceeds what would be expected from sampling error alone, suggesting true heterogeneity.
Answer The Q value of 15.2 indicates the magnitude of observed variation relative to expected error. If this value is statistically significant, it suggests that the variation in effect sizes is not just due to random chance but reflects true heterogeneity across the studies.
Common mistakes
- Confusing observed variation with true heterogeneity; observed variation includes both true heterogeneity and random error.
- Thinking that the absolute value of dispersion matters for isolating true variation; only the ratio of observed to expected dispersion is relevant.