Introduction to the Law of Large Numbers:
The Law of Large Numbers (LLN) is one of the fundamental theorems in probability theory. It establishes that when a random experiment is repeated a sufficiently large number of times, the relative frequency with which an event occurs tends to converge to its theoretical probability.
Formulation
Consider an experiment where each trial has a success probability p. For instance, when flipping a fair coin, p = 0.5 for getting heads. If we perform n independent trials and record the number of successes, the relative frequency:
f(n) = (number of successes) / n
gradually approaches p as n increases.
Two Versions of the Law
- Weak Law of Large Numbers: For every ε > 0, the probability that |f(n) – p| > ε tends to zero as n → ∞
- Strong Law of Large Numbers: With probability 1, f(n) converges to p as n → ∞
What the Simulation Reveals
Behavior for Small n
At the beginning (few trials), the trajectories show:
- High variability: Relative frequencies oscillate significantly
- Wide dispersion: Different trajectories are distributed over a wide interval
- Flat histogram: The frequency distribution is spread out
Behavior for Large n
As n increases, we observe:
- Convergence: All trajectories converge to the red line (p)
- Variance reduction: Oscillations diminish
- Concentration: The histogram shows an increasingly concentrated distribution around p
Interactive Parameters
- p (probability): Adjust the theoretical probability to observe universal convergence across different values
- m (trajectories): Increase the number of paths to enhance the visualization of empirical distribution patterns
- n (trials): Extend the experiment length to clearly demonstrate the convergence process
Stages of Convergence
Law of Large Numbers – Visual proof
n = 10
High variability: The relative frequencies fluctuate significantly between extreme values (0.0 – 1.0).
Wide dispersion: The different trajectories are spread over a very large range.
n = 50
Initial trends: Patterns of convergence toward p begin to emerge.
Reduced variability: The fluctuations start to decrease, but are still noticeable.
n = 100
Visible convergence: Most of the trajectories approach p.
Reduced oscillations: Oscillations are substantially reduced and less pronounced.
n = 500
Clear convergence: Almost all trajectories are very close to p.
Minimal oscillations: Remaining fluctuations are minimal and almost imperceptible.
n = 1000
Stable convergence: All trajectories are concentrated around p.
Statistical stability: Further increases in n produce only minimal improvements.
Interpreting the Data
The simulation illustrates the principle that statistical regularity emerges from random processes. The law of large numbers ensures that the sample mean converges in probability to the expected value as the number of trials increases.
Conclusion
This simulation offers a visual proof of a fundamental law of probability: randomness gives rise to statistical order. While each individual experiment is random, the mean of many experiments converges deterministically to the theoretical probability.
The Law of Large Numbers gives us confidence that with enough data, observed results will reliably reflect underlying probabilities.