Content
Summary
The distribution of the sum of independent continuous random variables is determined by the convolution of their individual probability density functions. For any linear combination of random variables, the expected value is the linear combination of the expected values, and for independent variables, the variance of the sum is the sum of their variances.
Intuition
When you add two independent random variables, you are essentially combining their uncertainties. The resulting distribution is not just a simple addition of values but a blending of their shapes, mathematically described by convolution. This process integrates the product of one variable's density and the shifted density of the other. While the shape of the distribution changes via convolution, the central tendency (expected value) and spread (variance) follow simpler linear rules, provided the variables are independent.
Key equations
The convolution of two density functions f and g, which gives the density of the sum of independent random variables.
A linear combination of random variables.
The expected value of a linear combination of random variables.
Derivation
- Define a linear combination L as a function of random variables X_i with coefficients c_i.
- For the sum of two independent continuous random variables X and Y, the density of Z = X + Y is derived by integrating the product of the density of X and the shifted density of Y over all possible values.
- This integration process is formally defined as the convolution integral (f * g)(z).
- The expected value of L is calculated by applying the linearity of expectation, resulting in the sum of coefficients times individual expected values.
- For independent variables, the variance of the sum is the sum of the individual variances, as covariance terms are zero.
Worked example
Calculate the density of the sum Z = X + Y where X and Y are independent uniform random variables on [0, 1].
- Identify that X and Y are independent uniform random variables on [0, 1].
- Apply the convolution formula for the sum of independent variables.
- Evaluate the integral of the product of densities over the support.
- Determine the resulting piecewise function based on the range of z.
Answer The density of Z is f_Z(z) = z for 0 <= z <= 1 and 2 - z for 1 < z <= 2.
Common mistakes
- Assuming the density of the sum is the convolution of densities without verifying independence.
- Confusing mixtures (selecting a distribution) with sums (adding random variables).
- Applying the sum of variances rule to dependent variables without accounting for covariance.