Proof of the Weak Law of Large Numbers and its Generalization

Weak Law of Large Numbers (WLLN)

Let be a sequence of independent and identically distributed (i.i.d.) random variables with finite expected value . Define the sample mean as:

Then, converges to in probability as approaches infinity (denoted as ).

Formally, for any , we have:

Generalized WLLN

Let be a sequence of i.i.d. random variables such that

Then , where

Why it is a generalization?

Before proving the generalized version, let's see why it is indeed a generalization of the standard WLLN. We first show that, for any random variable , finite expected value implies the condition required by the generalized WLLN:

Let , then

As , converges to almost surely. Combining this with the fact that , which is integrable, we can apply the Dominated Convergence Theorem to get:

Squeeze Theorem then gives us:

We also have as a direct consequence of DCT. Hence, if the generalized WLLN holds, then the standard WLLN also holds.

To complete the discussion and assist understanding, we show that the converse is not necessarily true (i.e., the condition of the generalized WLLN is strictly weaker than the finite expectation condition). For example, consider a random variable satisfying:

Then

but

Proof of the Generalized WLLN

Proof Sketch

Denote

Our proof will proceed in two steps: - Show that . - Show that , where .

Step 1:

We consider when . This occurs if and only if there exists some such that . Thus,

Since , the Squeeze Theorem gives us:

which directly implies .

Step 2:

For any , we must show that

where

We will use Chebyshev's inequality to achieve this. Let , then

It sufficies to show that

Note that

Let

Then

Using the assumption , we get

Hence,

This completes the proof of the generalized WLLN.