Content
Summary
Arbitrary uniform priors can cause high posterior sensitivity when data is scarce, but this effect diminishes as more data is collected. Alternatively, incorporating background knowledge via power law priors provides a more realistic and robust estimation of company sizes.
Intuition
When we lack sufficient data, our choice of prior distribution heavily influences the result, especially if that prior is arbitrary like a uniform distribution. For example, assuming all company sizes are equally likely ignores economic reality where small firms are far more common than large ones. By either collecting more data points or using a power law prior that reflects the natural skew of company sizes, we can reduce this sensitivity and arrive at more reliable estimates.
Key equations
The probability mass function for a power law prior, where is the firm size and is a parameter near 1.
Derivation
- Arbitrary uniform priors assign equal probability to all values, which is unrealistic for company sizes.
- With limited data, this leads to high sensitivity; e.g., changing the upper bound from 500 to 2000 shifts the posterior mean from 207 to 552.
- Collecting more data causes posteriors from different priors to converge, reducing the impact of the prior.
- Alternatively, using a power law prior incorporates the empirical fact that many small companies exist and few large ones do.
- The power law PMF decreases as size increases, providing a more justified prior than the uniform distribution.
Worked example
Compare the posterior mean sensitivity in the locomotive problem using a uniform prior with different upper bounds.
- With one data point and an upper bound of 500, the posterior mean is 207.
- Increasing the upper bound to 2000 with the same single data point raises the posterior mean to 552.
- Adding two more data points (30 and 90) reduces the disparity: means become 152, 164, and 171 for bounds 500, 1000, and 2000 respectively.
Answer The posterior mean is highly sensitive to the prior bound with one data point (207 vs 552), but this sensitivity is mitigated with three data points (152, 164, 171).
Common mistakes
- Assuming a uniform prior is always appropriate due to its simplicity.
- Ignoring the impact of arbitrary bounds on the posterior when data is scarce.
- Failing to recognize that company sizes follow a power law rather than a uniform distribution.