Central Limit Theorem: A Special Continuous Probability Distributions

 

The Central Limit Theorem (CLT) is a fundamental concept in statistics that states that the sampling distribution of the sample mean (or sum) of a large number of independent, identically distributed (i.i.d.) random variables approaches a normal distribution, regardless of the shape of the original distribution. This theorem is particularly important because it allows statisticians to make inferences about population parameters even when the population distribution is not normal.

Central Limit Theorem

Statement of the CLT

If $X_1, X_2, \ldots, X_n$ are i.i.d. random variables with mean $\mu$ and variance $\sigma^2$ , then the standardized sum of these random variables:

$Z = \frac{\sum_{i=1}^n X_i - n\mu}{\sqrt{n}\sigma}$

approaches a standard normal distribution $N(0,1)$ as $n \to \infty$ .

Implications

Sample Mean: The sample mean $\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i$ will be approximately normally distributed for large $n$ , with mean $\mu$ and variance $\frac{\sigma^2}{n}$ .

Sample Sum: The sum $\sum_{i=1}^n X_i$ will be approximately normally distributed for large $n$ , with mean $n\mu$ and variance $n\sigma^2$ .

Examples and Numerical Illustrations

Example 1: Uniform Distribution

Suppose we have a uniform distribution $U(a, b)$ . For simplicity, let's consider $U(0, 1)$ :

• Mean $\mu = \frac{a + b}{2} = 0.5$
• Variance $\sigma^2 = \frac{(b - a)^2}{12} = \frac{1}{12}$

Numerical Illustration: Let's draw 10,000 samples of size $n = 30$ from this distribution and compute the sample mean for each sample. According to the CLT, these sample means should follow a normal distribution with mean $0.5$ and variance $\frac{1}{12 \times 30} = \frac{1}{360} \approx 0.00278$ .

Example 2: Exponential Distribution

Consider an exponential distribution with parameter $\lambda = 1$ :

• Mean $\mu = \frac{1}{\lambda} = 1$
• Variance $\sigma^2 = \frac{1}{\lambda^2} = 1$

Numerical Illustration: Let's draw 10,000 samples of size $n = 50$ from this distribution and compute the sample mean for each sample. According to the CLT, these sample means should follow a normal distribution with mean $1$ and variance $\frac{1}{50} = 0.02$ .

Example 3: Binomial Distribution

Consider a binomial distribution $B(n, p)$ . For large $n$ , the binomial distribution can be approximated by a normal distribution due to the CLT. Let's take $B(20, 0.5)$ :

• Mean $\mu = np = 20 \times 0.5 = 10$
• Variance $\sigma^2 = np(1 - p) = 20 \times 0.5 \times 0.5 = 5$

Numerical Illustration: If we take large samples of size $n = 100$ from this binomial distribution and compute the sample mean for each sample, these sample means should follow a normal distribution with mean $10$ and variance $\frac{5}{100} = 0.05$ .

Special Continuous Probability Distributions and CLT

1. Uniform Distribution

The uniform distribution is a continuous probability distribution with constant probability. The CLT states that the sum (or mean) of a large number of uniform random variables will approach a normal distribution.

2. Exponential Distribution

The exponential distribution is often used to model the time between events in a Poisson process. Despite its skewness, the CLT assures that the sum (or mean) of a large number of exponentially distributed random variables will approach a normal distribution.

3. Gamma Distribution

The gamma distribution is a two-parameter family of continuous probability distributions. The sum of gamma-distributed random variables, with the same shape parameter, is also gamma-distributed. However, the CLT ensures that for large sample sizes, the distribution of the sum (or mean) will be approximately normal.

Conclusion

The Central Limit Theorem is a powerful statistical tool that enables us to use the normal distribution as an approximation for the distribution of sample means or sums, regardless of the shape of the original population distribution. This makes it easier to apply inferential statistics and hypothesis testing in practical situations.

Understanding and applying the CLT with special continuous probability distributions like the uniform, exponential, and gamma distributions illustrates its versatility and importance in statistical analysis.