Content
Summary
Advanced composition provides a rigorous framework for analyzing cumulative privacy loss when multiple adaptive mechanisms are applied sequentially to the same dataset. Unlike naive linear summation, it yields a tighter bound where the privacy parameter epsilon grows sublinearly with the number of queries k. This approach is essential for iterative programs and large-scale multi-query settings.
Intuition
When you run multiple differentially private mechanisms on the same data, the total privacy loss accumulates. A naive approach simply adds up the privacy budgets, leading to a linear growth of times . However, advanced composition recognizes that privacy losses do not always add up perfectly in the worst-case scenario. Instead, the dominant term scales with the square root of , meaning the privacy degradation is much slower than linear addition would suggest. This sublinear growth makes it feasible to perform many queries, provided the failure probability is managed carefully, as it grows linearly with .
Key equations
The exact bound for the composed privacy parameter epsilon prime, accounting for both Gaussian-like accumulation and approximate DP properties.
Simplified approximation for small epsilon values, highlighting the square-root dependence on k.
Derivation
- Identify a sequence of mechanisms, each satisfying -differential privacy.
- Apply the Advanced Composition Theorem to determine the cumulative privacy loss.
- Recognize that the total failure probability becomes , growing linearly with .
- Calculate the new epsilon prime using the formula involving a square root term for the dominant error and a second term for approximate DP specifics.
- Simplify the equation for small epsilon by neglecting the insignificant second term, resulting in a bound proportional to the square root of k.
Worked example
Compare the privacy cost for queries using naive composition versus advanced composition with and .
- Naive composition sums the budget linearly: total epsilon = k * epsilon = 500 * 0.1 = 50.
- For advanced composition, use the simplified approximation since is small: .
- Calculate ln(1/10^-5) = ln(10^5) approx 11.51.
- Compute .
- Multiply by epsilon: 107.3 * 0.1 = 10.73.
- The advanced composition bound (approx 10.73) is significantly lower than the naive bound (50).
Answer Naive cost is 50; Advanced composition cost is approximately 10.73.
Common mistakes
- Assuming advanced composition is always better than naive composition.
- Ignoring that the failure probability grows linearly with .
- Using the simplified square-root approximation for large epsilon values.
- Applying advanced composition for small k (less than 37) where naive composition is tighter.