StatlectThe Digital Textbook
Index > Asymptotic theory

Almost sure convergence

This lecture introduces the concept of almost sure convergence. In order to understand this lecture, you should first understand the concepts of almost sure property and almost sure event, explained in the lecture entitled Zero-probability events, and the concept of pointwise convergence of a sequence of random variables, explained in the lecture entitled Pointwise convergence.

Table of Contents

Almost sure convergence of a sequence of random variables

Let [eq1] be a sequence of random variables defined on a sample space Omega. The concept of almost sure convergence (or a.s. convergence) is a slight variation of the concept of pointwise convergence. As we have seen, a sequence of random variables [eq1] is pointwise convergent if and only if the sequence of real numbers [eq3] is convergent for all omega in Omega. Achieving convergence for all omega in Omega is a very stringent requirement. Therefore, this requirement is usually weakened, by requiring the convergence of [eq4] for a large enough subset of Omega, and not necessarily for all omega in Omega. In particular, [eq5] is usually required to be a convergent sequence almost surely: if F is the set of all sample points omega for which the sequence [eq6] is convergent, its complement $F^{c}$ must be included in a zero-probability event:[eq7]In other words, almost sure convergence requires that the sequences [eq3] converge for all sample points omega in Omega, except, possibly, for a very small set $F^{c}$ of sample points ($F^{c}$ must be included in a zero-probability event). This is summarized by the following definition.

Definition Let [eq1] be a sequence of random variables defined on a sample space Omega. We say that [eq1] is almost surely convergent (a.s. convergent) to a random variable X defined on Omega if and only if the sequence of real numbers [eq11] converges to [eq12] almost surely, i.e., if and only if there exists a zero-probability event E such that[eq13]X is called the almost sure limit of the sequence and convergence is indicated by[eq14]

The following is an example of a sequence that converges almost surely.

Example Suppose the sample space Omega is[eq15]It is possible to build a probability measure $QTR{rm}{P}$ on Omega, such that $QTR{rm}{P}$ assigns to each sub-interval of $left[ 0,1
ight] $ a probability equal to its length:[eq16](see the lecture entitled Zero-probability events). Remember that in this probability model all the sample points omega in Omega are assigned zero probability (each sample point, when considered as an event, is a zero-probability event):[eq17]Now, consider a sequence of random variables [eq1] defined as follows:[eq19]When [eq20], the sequence of real numbers [eq3] converges to 0 because[eq22]However, when $omega =0$, the sequence of real numbers [eq23] is not convergent to 0 because[eq24]Define a constant random variable X as follows: [eq25]We have that[eq26]But [eq27] because [eq28]which means that the event[eq29]is a zero-probability event. Therefore, the sequence [eq1] converges to X almost surely. Note, however, that [eq1] does not converge pointwise to X because [eq32] does not converge to [eq33] for all omega in Omega.

Almost sure convergence of a sequence of random vectors

The above notion of convergence generalizes to sequences of random vectors in a straightforward manner.

Let [eq1] be a sequence of random vectors defined on a sample space Omega, where each random vector X_n has dimension Kx1. Also in the case of random vectors, the concept of almost sure convergence is obtained from the concept of pointwise convergence by relaxing the assumption that the sequence [eq35] converges for all omega in Omega. Remember that the sequence of real vectors [eq36] converges to a real vector [eq33] if and only if [eq38] Instead, it is required that the sequence [eq39] converges for almost all omega (i.e., almost surely).

Definition Let [eq1] be a sequence of random vectors defined on a sample space Omega. We say that [eq1] is almost surely convergent to a random vector X defined on Omega if and only if the sequence of real vectors [eq39] converges to the real vector [eq33] almost surely, i.e., if and only if there exists a zero-probability event E such that[eq13]X is called the almost sure limit of the sequence and convergence is indicated by[eq14]

Now, denote by [eq46] the sequence of the i-th components of the vectors X_n. It can be proved that the sequence of random vectors [eq1] is almost surely convergent if and only if all the K sequences of random variables [eq46] are almost surely convergent.

Proposition Let [eq1] be a sequence of random vectors defined on a sample space Omega. Denote by [eq46] the sequence of random variables obtained by taking the i-th component of each random vector X_n. The sequence [eq1] converges almost surely to the random vector X if and only if [eq46] converges almost surely to the random variable $X_{ullet ,i}$ (the i-th component of X) for each $i=1,ldots ,K$.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let the sample space be[eq53]i.e. the sample space is the set of all real numbers between 0 and 1. Sub-intervals of $left[ 0,1
ight] $ are assigned a probability equal to their length:[eq54]

Define a sequence of random variables [eq1] as follows:[eq56]

Define a random variable X as follows:[eq57]

Does the sequence [eq1] converge almost surely to X?

Solution

For a fixed sample point [eq59], the sequence of real numbers [eq60] has limit[eq61]

For $omega =1$, the sequence of real numbers [eq39] has limit[eq63]

Therefore, the sequence of random variables [eq1] does not converge pointwise to X because[eq65]for $omega =1$. However, the set of sample points omega such that [eq66] does not converge to [eq12] is a zero-probability event: [eq68]Therefore, the sequence [eq1] converges almost surely to X.

Exercise 2

Let [eq1] and [eq71] be two sequences of random variables defined on a sample space Omega. Let X and Y be two random variables defined on Omega such that[eq72]

Prove that[eq73]

Solution

Denote by $F_{X}$ the set of sample points for which [eq74] converges to [eq33]:[eq76]The fact that [eq1] converges almost surely to X implies that[eq78]where [eq79].

Denote by $F_{Y}$ the set of sample points for which [eq80] converges to [eq81]:[eq82]The fact that [eq71] converges almost surely to Y implies that[eq84]where [eq85].

Now, denote by $F_{XY}$ the set of sample points for which [eq86] converges to [eq87]:[eq88]

Observe that if [eq89] then [eq90] converges to [eq91], because the sum of two sequences of real numbers is convergent if the two sequences are convergent. Therefore,[eq92]Taking the complement of both sides, we obtain[eq93]But [eq94]and as a consequence [eq95]. Thus, the set $F_{XY}^{c}$ of sample points omega such that [eq90] does not converge to [eq97] is included in the zero-probability event $E_{X}cup E_{Y}$, which means that[eq73]

Exercise 3

Let the sample space be[eq99]that is, the sample space is the set of all real numbers between 0 and 1. Sub-intervals of $left[ 0,1
ight] $ are assigned a probability equal to their length:[eq54]

Define a sequence of random variables [eq1] as follows:[eq102]

Find an almost sure limit of the sequence.

Solution

If $omega =0$ or $omega =1$, then the sequence of real numbers [eq3] is not convergent:[eq104]

For [eq105], the sequence of real numbers [eq3] has limit[eq107]because for any omega we can find $n_{0}$ such that [eq108] for any $ngeq n_{0}$ (as a consequence [eq109] for any $ngeq n_{0}$).

Thus, the sequence of random variables [eq1] converges almost surely to the random variable X defined as[eq111]because the set of sample points omega such that [eq112] does not converge to [eq33] is a zero-probability event: [eq114]

The book

Most of the learning materials found on this website are now available in a traditional textbook format.