Search for probability and statistics terms on Statlect
StatLect
Index > Asymptotic theory

Mean-square convergence

by , PhD

This lecture discusses mean-square convergence, first for sequences of random variables and then for sequences of random vectors.

The formula that defines convergence in mean-square and its main elements.

Table of Contents

The intuition

As explained previously, different definitions of convergence are based on different ways of measuring how similar to each other two random variables are.

The definition of mean-square convergence is based on the following intuition: two random variables are similar to each other if the square of their difference is small on average.

How to measure similarity

Remember that a random variable is a mapping from a sample space Omega (e.g., the set of possible outcomes of a coin-flipping experiment) to the set of real numbers (e.g., the winnings from betting on tails).

Let X_n and X be random variables defined on the same sample space Omega.

For a fixed sample point omega in Omega, the squared difference [eq1]between the two realizations of X_n and X provides a measure of how different those two realizations are.

The mean squared difference [eq2]quantifies how different the two realizations are on average (as omega varies).

It is a measure of the "distance" between the two variables. In technical terms, it is called a metric.

How to define convergence

Intuitively, if a sequence [eq3] converges to X, the mean squared difference should become smaller and smaller by increasing n.

In other words, the sequence of real numbers[eq4]should converge to zero.

Requiring that a sequence of distances tends to zero is a standard criterion for convergence in a metric space.

Square integrability

This kind of convergence analysis can be carried out only if the expected values of $X^{2}$ and $X_{n}^{2}$ are well-defined and finite.

In technical terms, we say that X and X_n are required to be square integrable.

Definition for sequences of random variables

The considerations above lead us to define mean-square convergence as follows.

Definition Let [eq5] be a sequence of square integrable random variables defined on a sample space Omega. We say that [eq6] is mean-square convergent (or convergent in mean-square) if and only if there exists a square integrable random variable X such that[eq7]

The variable X is called the mean-square limit of the sequence and convergence is indicated by[eq8]or by[eq9]

The notation [eq10] indicates that convergence is in the Lp space $L^{2}$ (the space of square integrable functions).

Example

The following example illustrates the concept of mean-square convergence.

Let [eq11] be a covariance stationary sequence of random variables such that all the random variables in the sequence have:

Define the sample mean Xbar_n as follows:[eq12]and define a constant random variable $X=mu $.

The distance between a generic term of the sequence [eq13] and X is[eq14]

But mu is equal to the expected value of Xbar_n because[eq15]Therefore,[eq16]by the very definition of variance.

In turn, the variance of Xbar_n is[eq17]

Thus,[eq18]and [eq19]

But this is just the definition of mean square convergence of Xbar_n to X.

Therefore, the sequence [eq20] converges in mean-square to the constant random variable $X=mu $.

How to generalize the definition to the multivariate case

The above notion of convergence generalizes to sequences of random vectors in a straightforward manner.

Let [eq21] be a sequence of random vectors defined on a sample space Omega, where each random vector X_n has dimension Kx1.

The sequence of random vectors [eq22] is said to converge to a random vector X in mean-square if [eq23] converges to X according to the metric [eq24]where [eq25] is the Euclidean norm of the difference between X_n and X and the second subscript is used to indicate the individual components of the vectors X_n and X.

The distance [eq26] is well-defined only if the expected value on the right-hand side exists. A sufficient condition for its existence is that all the components of X_n and X be square integrable random variables.

Intuitively, for a fixed sample point omega, the square of the Euclidean norm [eq27] provides a measure of the distance between two realizations of X_n and $X $.

The mean [eq28] provides a measure of how different those two realizations are on average (as omega varies).

If the distance becomes smaller and smaller by increasing n, then the sequence of random vectors [eq29] converges to the vector X.

Definition for sequences of random vectors

The following definition formalizes what we have just said.

Definition Let [eq30] be a sequence of random vectors defined on a sample space Omega, whose entries are square integrable random variables. We say that [eq31] is mean-square convergent if and only if there exists a random vector X with square integrable entries such that[eq32]

Again, X is called the mean-square limit of the sequence and convergence is indicated by[eq33]or by[eq9]

Relation between multivariate and univariate convergence

A sequence of random vectors is convergent in mean-square if and only if all the sequences of entries of the random vectors are.

Proposition Let [eq35] be a sequence of random vectors defined on a sample space Omega, such that their entries are square integrable random variables. Denote by [eq36] the sequence of random variables obtained by taking the i-th entry of each random vector X_n. The sequence [eq37] converges in mean-square to the random vector X if and only if [eq38] converges in mean-square to the random variable $X_{ullet ,i}$ (the i-th entry of X) for each $i=1,ldots ,K$.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let $U$ be a random variable having a uniform distribution on the interval $left[ 1,2 ight] $.

In other words, $U$ is a continuous random variable with support[eq39]and probability density function[eq40]

Consider a sequence of random variables [eq41] whose generic term is[eq42]where [eq43] is the indicator function of the event [eq44].

Find the mean-square limit (if it exists) of the sequence [eq45].

Solution

When n tends to infinity, the interval [eq46] becomes similar to the interval $left[ 1,2 ight] $ because[eq47]Therefore, we conjecture that the indicators [eq48] converge in mean-square to the indicator [eq49]. But [eq50] is always equal to 1, so our conjecture is that the sequence [eq51] converges in mean square to 1. To verify our conjecture, we need to verify that[eq52]The expected value can be computed as follows.[eq53]Thus, the sequence [eq41] converges in mean-square to 1 because[eq55]

Exercise 2

Let [eq41] be a sequence of discrete random variables.

Let the probability mass function of a generic term of the sequence X_n be[eq57]

Find the mean-square limit (if it exists) of the sequence [eq45].

Solution

Note that[eq59]Therefore, one would expect that the sequence [eq41] converges to the constant random variable $X=0$. However, the sequence [eq61] does not converge in mean-square to 0. The distance of a generic term of the sequence from 0 is[eq62]Thus,[eq63]while, if [eq41] was convergent, we would have[eq65]

Exercise 3

Does the sequence in the previous exercise converge in probability?

Solution

The sequence [eq41] converges in probability to the constant random variable $X=0$ because, for any $ arepsilon >0$, we have that[eq67]

How to cite

Please cite as:

Taboga, Marco (2021). "Mean-square convergence", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/mean-square-convergence.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.