StatlectThe Digital Textbook
Index > Asymptotic theory

Mean-square convergence

This lecture discusses mean-square convergence. We deal first with mean-square convergence of sequences of random variables and then with mean-square convergence of sequences of random vectors.

Mean-square convergence of a sequence of random variables

In the lecture entitled Sequences of random variables and their convergence we have stressed the fact that different concepts of convergence are based on different ways of measuring the distance between two random variables (how "close to each other" two random variables are). The concept of mean-square convergence, or convergence in mean-square, is based on the following intuition: two random variables are "close to each other" if the square of their difference is on average small.

Let [eq1] be a sequence of random variables defined on a sample space Omega. Let X be a random variable. The sequence [eq2] is said to converge to X in mean-square if [eq1] converges to X according to the metric [eq4] defined as follows:[eq5](if you do not understand what it means "to converge according to a metric" go to the lecture entitled Limit of a sequence).

Note that [eq6] is well-defined only if the expected value on the right hand side exists, which is usually ensured by requiring that X_n and X be square integrable.

Intuitively, for a fixed sample point omega, the squared difference [eq7] between the two realizations of X_n and X provides a measure of how different those two realizations are. The mean squared difference [eq8] provides a measure of how different those two realizations are on average (as omega varies). If this mean squared difference becomes smaller and smaller by increasing n, then the sequence [eq1] converges to X.

We summarize the concept of mean-square convergence in the following definition.

Definition Let [eq1] be a sequence of square integrable random variables defined on a sample space Omega. We say that [eq11] is mean-square convergent (or convergent in mean-square) if and only if there exists a square integrable random variable X such that [eq1] converges to X, according to the metric [eq13], that is,[eq14]X is called the mean-square limit of the sequence and convergence is indicated by[eq15]or by[eq16]

Note that in the definition above, [eq17] is just the usual criterion for convergence, while [eq18] indicates that convergence is in the Lp space $L^{2}$, because both [eq1] and X have been required to be square integrable.

The following example illustrates the concept of mean-square convergence.

Example Let [eq1] be a covariance stationary sequence of random variables such that all the random variables in the sequence have the same expected value mu, the same variance sigma^2 and zero covariance with each other. Define the sample mean Xbar_n as follows:[eq21]and define a constant random variable $X=mu $. The distance between a generic term of the sequence [eq22] and X is[eq23]But mu is equal to the expected value of Xbar_n because[eq24]Therefore,[eq25]by the very definition of variance. In turn, the variance of Xbar_n is[eq26]Thus,[eq27]and [eq28]But this is just the definition of mean square convergence of Xbar_n to X. Therefore, the sequence [eq29] converges in mean-square to the constant random variable $X=mu $.

Mean-square convergence of a sequence of random vectors

The above notion of convergence generalizes to sequences of random vectors in a straightforward manner.

Let [eq1] be a sequence of random vectors defined on a sample space Omega, where each random vector X_n has dimension Kx1. The sequence of random vectors [eq1] is said to converge to a random vector X in mean-square if [eq1] converges to X according to the metric [eq4] defined as follows:[eq34]where [eq35] is the Euclidean norm of the difference between X_n and X and the second subscript is used to indicate the individual components of the vectors X_n and X.

Of course, [eq6] is well-defined only if the expected value on the right hand side exists. A sufficient condition for [eq37] to be well-defined is that all the components of X_n and X be square integrable random variables.

Intuitively, for a fixed sample point omega, the square of the Euclidean norm [eq38] of the difference between the two realizations of X_n and X provides a measure of how different those two realizations are. The mean of the square of the Euclidean norm [eq39] provides a measure of how different those two realizations are on average (as omega varies). If [eq40] becomes smaller and smaller by increasing n, then the sequence of random vectors [eq1] converges to the vector X.

The following is a formal definition of mean-square convergence for random vectors.

Definition Let [eq1] be a sequence of random vectors defined on a sample space Omega, whose components are square integrable random variables. We say that [eq1] is mean-square convergent (or convergent in mean-square) if and only if there exists a random vector X with square integrable components such that [eq1] converges to X, according to the metric [eq45], that is,[eq46]X is called the mean-square limit of the sequence and convergence is indicated by[eq15]or by[eq16]

Note that in the definition above, [eq49] is just the usual criterion for convergence, while [eq18] indicates that convergence is in the Lp space $L^{2}$, because both [eq51] and X have been required to have square integrable components.

Now, denote by [eq52] the sequence of the i-th components of the vectors X_n. It can be proved that the sequence of random vectors [eq1] is convergent in mean-square if and only if all the K sequences of random variables [eq54] are convergent in mean-square.

Proposition Let [eq1] be a sequence of random vectors defined on a sample space Omega, such that their components are square integrable random variables. Denote by [eq56] the sequence of random variables obtained by taking the i-th component of each random vector X_n. The sequence [eq1] converges in mean-square to the random vector X if and only if [eq58] converges in mean-square to the random variable $X_{ullet ,i}$ (the i-th component of X) for each $i=1,ldots ,K$.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let $U$ be a random variable having a uniform distribution on the interval $left[ 1,2
ight] $. In other words, $U$ is an absolutely continuous random variable with support[eq59]and probability density function[eq60]Consider a sequence of random variables [eq1] whose generic term is[eq62]where [eq63] is the indicator function of the event [eq64].

Find the mean-square limit (if it exists) of the sequence [eq65].

Solution

When n tends to infinity, the interval [eq66] becomes similar to the interval $left[ 1,2
ight] $ because[eq67]Therefore, we conjecture that the indicators [eq68] converge in mean-square to the indicator [eq69]. But [eq70] is always equal to 1, so our conjecture is that the sequence [eq71] converges in mean square to 1. To verify our conjecture, we need to verify that[eq72]

The expected value can be computed as follows.[eq73]Thus, the sequence [eq1] converges in mean-square to 1 because[eq75]

Exercise 2

Let [eq1] be a sequence of discrete random variables. Let the probability mass function of a generic term of the sequence X_n be[eq77]

Find the mean-square limit (if it exists) of the sequence [eq65].

Solution

Note that[eq79]Therefore, one would expect that the sequence [eq1] converges to the constant random variable $X=0$. However, the sequence [eq81] does not converge in mean-square to 0. The distance of a generic term of the sequence from 0 is[eq82]Thus,[eq83]while, if [eq1] was convergent, we would have[eq85]

Exercise 3

Does the sequence in the previous exercise converge in probability?

Solution

The sequence [eq1] converges in probability to the constant random variable $X=0$ because, for any $arepsilon >0$, we have that[eq87]

The book

Most learning materials found on this website are now available in a traditional textbook format.