This lecture introduces the concept of almost sure convergence. In order to understand this lecture, you should first understand the concepts of almost sure property and almost sure event, explained in the lecture entitled Zero-probability events, and the concept of pointwise convergence of a sequence of random variables, explained in the lecture entitled Pointwise convergence.

Let
be a sequence of random variables defined on a
sample space
.
The concept of **almost sure convergence** (or **a.s.
convergence**) is a slight variation of the concept of pointwise
convergence. As we have seen, a sequence of random variables
is pointwise convergent if and only if the sequence of real numbers
is convergent for all
.
Achieving convergence for all
is a very stringent requirement. Therefore, this requirement is usually
weakened, by requiring the convergence of
for a large enough subset of
,
and not necessarily for all
.
In particular,
is usually required to be a convergent sequence
almost surely: if
is the set of all sample points
for which the sequence
is convergent, its complement
must be included in a zero-probability
event:In
other words, almost sure convergence requires that the sequences
converge for all sample points
,
except, possibly, for a very small set
of sample points
(
must be included in a zero-probability event). This is summarized by the
following definition.

Definition
Let
be a sequence of random variables defined on a sample space
.
We say that
is **almost surely convergent** (**a.s.
convergent**) to a random variable
defined on
if and only if the sequence of real numbers
converges to
almost surely, i.e., if and only if there exists a zero-probability event
such
that
is called the **almost sure limit** of the sequence and
convergence is indicated
by

The following is an example of a sequence that converges almost surely.

Example Suppose the sample space isIt is possible to build a probability measure on , such that assigns to each sub-interval of a probability equal to its length:(see the lecture entitled Zero-probability events). Remember that in this probability model all the sample points are assigned zero probability (each sample point, when considered as an event, is a zero-probability event):Now, consider a sequence of random variables defined as follows:When , the sequence of real numbers converges to becauseHowever, when , the sequence of real numbers is not convergent to becauseDefine a constant random variable as follows: We have thatBut because which means that the eventis a zero-probability event. Therefore, the sequence converges to almost surely. Note, however, that does not converge pointwise to because does not converge to for all .

The above notion of convergence generalizes to sequences of random vectors in a straightforward manner.

Let be a sequence of random vectors defined on a sample space , where each random vector has dimension . Also in the case of random vectors, the concept of almost sure convergence is obtained from the concept of pointwise convergence by relaxing the assumption that the sequence converges for all . Remember that the sequence of real vectors converges to a real vector if and only if Instead, it is required that the sequence converges for almost all (i.e., almost surely).

Definition
Let
be a sequence of random vectors defined on a sample space
.
We say that
is **almost surely convergent** to a random vector
defined on
if and only if the sequence of real vectors
converges to the real vector
almost surely, i.e., if and only if there exists a zero-probability event
such
that
is called the **almost sure limit** of the sequence and
convergence is indicated
by

Now, denote by the sequence of the -th components of the vectors . It can be proved that the sequence of random vectors is almost surely convergent if and only if all the sequences of random variables are almost surely convergent.

Proposition Let be a sequence of random vectors defined on a sample space . Denote by the sequence of random variables obtained by taking the -th component of each random vector . The sequence converges almost surely to the random vector if and only if converges almost surely to the random variable (the -th component of ) for each .

Below you can find some exercises with explained solutions.

Let the sample space bei.e. the sample space is the set of all real numbers between 0 and 1. Sub-intervals of are assigned a probability equal to their length:

Define a sequence of random variables as follows:

Define a random variable as follows:

Does the sequence converge almost surely to ?

Solution

For a fixed sample point , the sequence of real numbers has limit

For , the sequence of real numbers has limit

Therefore, the sequence of random variables does not converge pointwise to becausefor . However, the set of sample points such that does not converge to is a zero-probability event: Therefore, the sequence converges almost surely to .

Let and be two sequences of random variables defined on a sample space . Let and be two random variables defined on such that

Prove that

Solution

Denote by the set of sample points for which converges to :The fact that converges almost surely to implies thatwhere .

Denote by the set of sample points for which converges to :The fact that converges almost surely to implies thatwhere .

Now, denote by the set of sample points for which converges to :

Observe that if then converges to , because the sum of two sequences of real numbers is convergent if the two sequences are convergent. Therefore,Taking the complement of both sides, we obtainBut and as a consequence . Thus, the set of sample points such that does not converge to is included in the zero-probability event , which means that

Let the sample space bethat is, the sample space is the set of all real numbers between 0 and 1. Sub-intervals of are assigned a probability equal to their length:

Define a sequence of random variables as follows:

Find an almost sure limit of the sequence.

Solution

If or , then the sequence of real numbers is not convergent:

For , the sequence of real numbers has limitbecause for any we can find such that for any (as a consequence for any ).

Thus, the sequence of random variables converges almost surely to the random variable defined asbecause the set of sample points such that does not converge to is a zero-probability event:

The book

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Point estimation
- Multivariate normal distribution
- Hypothesis testing
- Maximum likelihood
- Bayes rule
- Gamma function

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

- Alternative hypothesis
- Power function
- Posterior probability
- Discrete random variable
- Type I error
- Loss function

Share