Proof of the Strong Law of Large Numbers (SLLN) using Kronecker's lemma, Kolmogorov's maximal inequality, and Khinchine-Kolmogorov convergence theorem.
Preliminaries
Kronecker's Lemma. Let be a sequence of real numbers
such that
Then, for any such that
and
, we have
Proof. Define . Then,
Kolmogorov's Maximal Inequality. Let be a sequence of
independent random variables with zero expectations
and finite variances, and define . Then, for any
,
Proof. Let (with the convention that ). Then,
Khinchine-Kolmogorov Convergence Theorem. Let be a sequence of independent
random variables with zero expectations and finite
variances. If , then the series converges almost
surely.
Proof. For any and any ,
pick such that . For any , Kolmogorov's maximal inequality implies that
Then, apply the continuity of probability measure to get
Since
for any , we
have
This means that, for any , we can find such that
the tail sums are uniformly bounded by with probability at least . Hence, for almost all in the probability space , we can find
such that the tail sums
after are uniformly
bounded by .
Formally, there exists a 1-measure set such that, for any ,
Let
Then is also a 1-measure set,
and for any and any
, it holds that
Hence converges by the Cauchy criterion. Therefore, converges almost
surely.
Strong Law of Large Numbers
Theorem Statement
The Strong Law of Large Numbers (SLLN). Let be a sequence of independent
and identically distributed (i.i.d.) random variables with
finite expectation. Define , then almost surely.
Decomposition
Let
Evidently, . If we can show that and , then follows.
Proving
Since are independent
with zero expectations and finite variances, if we can show that
then
converges almost surely by the Khinchine-Kolmogorov convergence
theorem. Then, Kronecker's lemma implies that
where .
Now we show that :
This completes the proof that .
Proving
First, note that
By the Borel-Cantelli lemma, we have
Hence, for almost all in
the probability space , there exists only finite many such that .
Consequently,
Lastly, Dominated Convergence Theorem implies that
Combining the two results above, we conclude that .