Search for probability and statistics terms on Statlect
StatLect

Limit of a sequence

by , PhD

In this lecture we introduce the notion of limit of a sequence [eq1]. We start from the simple case in which [eq2] is a sequence of real numbers, then we deal with the general case in which [eq1] can be a sequence of objects that are not necessarily real numbers.

Table of Contents

The limit of a sequence of real numbers

We first give an informal definition and then a more formal definition of the limit of a sequence of real numbers.

Informal definition of the limit of a sequence of real numbers

Let [eq1] be a sequence of real numbers. Let $n_{0}in U{2115} $. Denote by [eq5] a subsequence of [eq2] obtained by dropping the first $n_{0}$ terms of [eq2], i.e.,[eq8]The following is an intuitive definition of limit of a sequence.

Definition (informal) Let a be a real number. We say that a is a limit of a sequence [eq9] of real numbers if, by appropriately choosing $n_{0}$, the distance between a and any term of the subsequence [eq10] can be made as close to zero as we like. If a is a limit of the sequence [eq1], we say that the sequence [eq12] is a convergent sequence and that it converges to a. We indicate the fact that a is a limit of [eq12] by[eq14]

Thus, a is a limit of [eq1] if, by dropping a sufficiently high number of initial terms of [eq1], we can make the remaining terms of [eq1] as close to a as we like. Intuitively, a is a limit of [eq1] if a_n becomes closer and closer to a by letting n go to infinity.

Formal definition of the limit of a sequence of real numbers

The distance between two real numbers is the absolute value of their difference. For example, if $ain U{211d} $ and a_n is a term of a sequence [eq1], the distance between a_n and a, denoted by [eq20], is[eq21]By using the concept of distance, the above informal definition can be made rigorous.

Definition (formal) Let $ain U{211d} $. We say that a is a limit of a sequence [eq1] of real numbers if[eq23]If a is a limit of the sequence [eq1], we say that the sequence [eq1] is a convergent sequence and that it converges to a. We indicate the fact that a is a limit of [eq12] by[eq14]

For those unfamiliar with the universal quantifiers $orall $ (any) and $exists $ (exists), the notation [eq23]reads as follows: "For any arbitrarily small number epsilon, there exists a natural number $n_{0}$ such that the distance between a_n and $a $ is less than epsilon for all the terms a_n with $n>n_{0}$", which can also be restated as "For any arbitrarily small number epsilon, you can find a subsequence [eq29] such that the distance between any term of the subsequence and a is less than epsilon" or as "By dropping a sufficiently high number of initial terms of [eq1], you can make the remaining terms as close to a as you wish".

It is also possible to prove that a convergent sequence has a unique limit, i.e., if [eq1] has a limit a, then a is the unique limit of [eq1].

Example Define a sequence [eq1] by characterizing its n-th element a_n as follows:[eq34]The elements of the sequence are $a_{1}=1$, $frac{1}{2}$, $frac{1}{3}$, $frac{1}{4}$ and so on. The higher n is, the smaller a_n is and the closer it gets to 0. Therefore, intuitively, the limit of the sequence should be 0:[eq35]It is straightforward to prove that 0 is indeed a limit of [eq2] by using the above definition. Choose any $arepsilon >0$. We need to find an $n_{0}in U{2115} $ such that all terms of the subsequence [eq37] have distance from zero less than epsilon:[eq38]Note first that the distance between a generic term of the sequence a_n and 0 is[eq39]where the last equality obtains from the fact that all the terms of the sequence are positive (hence they are equal to their absolute values). Therefore, we need to find an $n_{0}in U{2115} $ such that all terms of the subsequence [eq40] satisfy[eq41]Since[eq42]the condition[eq41]is satisfied if [eq44], which is equivalent to [eq45]. Therefore, it suffices to pick any $n_{0}$ such that [eq46] to satisfy the condition[eq47]In summary, we have just shown that, for any epsilon, we are able to find $n_{0}in U{2115} $ such that all terms of the subsequence [eq40] have distance from zero less than epsilon. As a consequence 0 is the limit of the sequence [eq1].

The limit of a sequence in general

We now deal with the more general case in which the terms of the sequence [eq12] are not necessarily real numbers. As before, we first give an informal definition, then a more formal one.

Informal definition of limit - The general case

Let A be a set of objects (e.g., real numbers, events, random variables) and let [eq1] be a sequence of elements of A. The limit of [eq1] is defined as follows.

Definition (informal) Let $ain A$. We say that a is a limit of a sequence [eq2] of elements of A, if, by appropriately choosing $n_{0}$, the distance between a and any term of the subsequence [eq10] can be made as close to zero as we like. If a is a limit of the sequence [eq1], we say that the sequence [eq12] is a convergent sequence and that it converges to a. We indicate the fact that a is a limit of [eq12] by[eq14]

The definition is the same we gave above, except for the fact that now both a and the terms of the sequence [eq1] belong to a generic set of objects A.

Metrics and the definition of distance

In the definition above, we have implicitly assumed that the concept of distance between elements of A is well-defined. Thus, for the above definition to make any sense, we need to properly define distance.

We need a function [eq60] that associates to any couple of elements of A a real number measuring how far these two elements are. For example, if a and $a^{prime }$ are two elements of A, [eq61] needs to be a real number measuring the distance between a and $a^{prime }$.

A function [eq62] is considered a valid distance function (and it is called a metric on A) if it satisfies some properties, listed in the next proposition.

Definition Let A be a set of objects. Let [eq63]. $d$ is considered a valid distance function (in which case it is called a metric on A) if, for any a, $a^{prime }$ and $a^{prime prime }$ belonging to A:

  1. non-negativity: [eq64];

  2. identity of indiscernibles: [eq65] if and only if $a=a^{prime }$;

  3. symmetry: [eq66];

  4. triangle inequality: [eq67].

All four properties are very intuitive: property 1) says that the distance between two points cannot be a negative number; property 2) says that the distance between two points is zero if and only if the two points coincide; property 3) says that the distance from a to $a^{prime }$ is the same as the distance from $a^{prime }$ to a; property 4) says (roughly speaking) that the distance you cover when you go from a to $a^{prime prime }$ directly is less than (or equal to) the distance you cover when you go from a to $a^{prime prime }$ passing from a third point $a^{prime }$ (if $a^{prime }$ is not on the way from a to $a^{prime prime }$ you are increasing the distance covered).

Example (Euclidean distance) Consider the set of K-dimensional real vectors $U{211d} ^{K}$. The metric usually employed to measure the distance between elements of $U{211d} ^{K}$ is the so-called Euclidean distance. If a and $b$ are two vectors belonging to $U{211d} ^{K}$, then their Euclidean distance is[eq68]where [eq69] are the K components of a and [eq70] are the K components of $b$. It is possible to prove that the Euclidean distance satisfies all the four properties that a metric needs to satisfy. Furthermore, when $K=1$, it becomes[eq71]which coincides with the definition of distance between real numbers already given above.

Whenever we are faced with a sequence of objects and we want to assess whether it is convergent, we need to first define a distance function on the set of objects to which the terms of the sequence belong and verify that the proposed distance function satisfies all the properties of a proper distance function (a metric). For example, in probability theory and statistics, we often deal with sequences of random variables. To assess whether these sequences are convergent, we need to define a metric to measure the distance between two random variables. As we will see in the lecture entitled Sequences of random variables and their convergence, there are several ways of defining the concept of distance between two random variables. All these ways are legitimate and are useful in different situations.

Formal definition of limit - The general case

Having defined the concept of a metric, we are now ready to state the formal definition of a limit of a sequence.

Definition (formal) Let A be a set of objects. Let [eq72] be a metric on A. We say that $ain A$ is a limit of a sequence [eq1] of objects belonging to A if[eq23]If a is a limit of the sequence [eq1], we say that the sequence [eq1] is a convergent sequence and that it converges to a. We indicate the fact that a is a limit of [eq12] by[eq14]

Also in this case, it is possible to prove (see below) that a convergent sequence has a unique limit, i.e., if [eq1] has a limit $a $, then a is the unique limit of [eq1].

Proof

The proof is by contradiction. Suppose that a and $a^{prime }$ are two limits of a sequence [eq1] and $a
eq a^{prime }$. By combining property 1) and 2) of a metric (see above) it must be that[eq82]i.e., [eq83] where $overline{d}$ is a strictly positive constant. Pick any term a_n of the sequence. By property 4) of a metric (the triangle inequality), we have[eq84]Considering that [eq83], the previous inequality becomes[eq86]Now, take any [eq87]. Since a is a limit of the sequence, we can find $n_{0}$ such that [eq88], which means that[eq89]and[eq90]Therefore, [eq91] can not be made smaller than [eq92] and as a consequence $a^{prime }$ cannot be a limit of the sequence.

Convergence criterion

In practice, it is usually difficult to assess the convergence of a sequence using the above definition. Instead, convergence can be assessed using the following criterion.

Lemma (criterion for convergence) Let A be a set of objects. Let [eq93] be a metric on A. Let [eq1] be a sequence of objects belonging to A and $ain A$. [eq1] converges to a if and only if[eq96]

Proof

This is easily proved by defining a sequence of real numbers [eq97] whose generic term is[eq98]and noting that the definition of convergence of [eq1] to a, which is[eq23]can be written as[eq101]which is the definition of convergence of [eq102] to 0.

So, in practice, the problem of assessing the convergence of a generic sequence of objects is simplified as follows:

  1. find a metric [eq103] to measure the distance between the terms of the sequence a_n and the candidate limit a;

  2. define a new sequence [eq102], where [eq105];

  3. study the convergence of the sequence [eq102], which is a simple problem, because [eq102] is a sequence of real numbers.

How to cite

Please cite as:

Taboga, Marco (2021). "Limit of a sequence", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/mathematical-tools/limit-of-a-sequence.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.