This lecture discusses convergence in distribution, first for sequences of random variables and then for sequences of random vectors.
Table of contents
We have previously explained that different concepts of convergence are based on different ways of measuring the distance between two random variables (how "close to each other" two random variables are).
The concept of convergence in distribution is based on the following intuition: two random variables are "close to each other" if their distribution functions are "close to each other".
Let us consider a sequence of random variables and a generic random variable belonging to the sequence.
Denote by the distribution function of . is a function .
Once we fix , the following facts are true:
the value associated to the point is a real number;
the sequence is a sequence of real numbers;
we can easily assess whether the sequence is convergent by using the standard definition of convergence of a sequence of real numbers.
If the sequence is convergent, its limit depends on the specific we have fixed.
A sequence of random variables is said to be convergent in distribution if and only if the sequence is convergent for any choice of , except possibly for some "special values" of where is not continuous in .
The following definition summarizes what we have said above.
Definition Let be a sequence of random variables. Denote by the distribution function of . We say that is convergent in distribution (or convergent in law) if and only if there exists a distribution function such that the sequence converges to for all points where is continuous.
If a random variable has distribution function , then is called the limit in distribution (or limit in law) of the sequence and convergence is indicated by
Note that convergence in distribution only involves the distribution functions of the random variables belonging to the sequence and that these random variables need not be defined on the same sample space.
On the contrary, the modes of convergence we have discussed in previous lectures (pointwise convergence, almost sure convergence, convergence in probability, mean-square convergence) require that all the variables in the sequence be defined on the same sample space.
Let us make an example
Let be a sequence of IID random variables all having a uniform distribution on the interval .
In other words, the distribution function of is
Define
The distribution function of is
Thus,
Since we havewhere is the distribution function of an exponential random variable.
Therefore, the sequence converges in law to an exponential distribution.
Let be a sequence of random variables and denote by the distribution function of .
Suppose that we find a function such that for all where is continuous.
How do we check that is a proper distribution function, so that we can say that the sequence converges in distribution?
As explained in the glossary entry on distribution functions, we need to check that satisfies the four properties characterizing proper distribution functions, that is, must be increasing, right-continuous and its limits at minus and plus infinity must be and
The definition of convergence in distribution of a sequence of random vectors is almost identical to that given for random variables.
We just need to replace distribution functions in the above definition with joint distribution functions.
Definition Let be a sequence of random vectors. Denote by the joint distribution function of . We say that is convergent in distribution (or convergent in law) if and only if there exists a joint distribution function such that the sequence converges to for all points where is continuous.
If a random vector has joint distribution function , then is called the limit in distribution (or limit in law) of the sequence and convergence is indicated by
It is important to note that for other notions of stochastic convergence (in probability, almost sure and in mean-square), the convergence of each single entry of the random vector is necessary and sufficient for their joint convergence, that is, for the convergence of the vector as a whole. Instead, for convergence in distribution, the individual convergence of the entries of the vector is necessary but not sufficient for their joint convergence.
Below you can find some exercises with explained solutions.
Let be a sequence of random variables having distribution functions
Find the limit in distribution (if it exists) of the sequence .
If , thenIf , thenWe now need to verify that the functionis a proper distribution function. The function is increasing, continuous, its limit at minus infinity is and its limit at plus infinity is , hence it satisfies the four properties that a proper distribution function needs to satisfy. This implies that converges in distribution to a random variable having distribution function .
Let be a sequence of random variables having distribution functions
Find the limit in distribution (if it exists) of the sequence .
If , thenIf , thenTherefore, the distribution functions converge to the functionwhich is not a proper distribution function, because it is not right-continuous at the point . However, note that the function is a proper distribution function and it is equal to at all points except at the point . But this is a point of discontinuity of . As a consequence, the sequence converges in distribution to a random variable having distribution function .
Let be a sequence of random variables having distribution functions
Find the limit in distribution (if it exists) of the sequence .
The distribution functions converge to the functionThis is the same limiting function found in the previous exercise. As a consequence, the sequence converges in distribution to a random variable having distribution function
Please cite as:
Taboga, Marco (2021). "Convergence in distribution", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/convergence-in-distribution.
Most of the learning materials found on this website are now available in a traditional textbook format.