Let be a sequence of random variables such thatwhere is a normal distribution with mean and variance , is a constant, and indicates convergence in distribution. is said to be asymptotically normal, is called the asymptotic mean of and its asymptotic variance.

For example, could be a sequence of sample means that are asymptotically normal because a Central Limit Theorem applies. Or it could be a sequence of maximum likelihood estimators satisfying a set of conditions that are sufficient for asymptotic normality.

Now, consider the sequence where is a function. The delta method is a method that allows us to derive, under appropriate conditions, the asymptotic distribution of from the asymptotic distribution of . A formal statement of the delta method is given in the following proposition.

Proposition Let be a sequence of random variables such thatLet be a continuously differentiable function. Then,

Proof

By the mean value theorem, there exists a point lying between and , such thatBy subtracting from both sides and multiplying by , we obtainSince converges in probability to , and lies between and , also converges in probability to . Because the derivative is continuous, by the continuous mapping theoremwhere denotes convergence in probability. Therefore, the first term of the product converges in probability to a constant. By assumption, the second term converges in distribution to a normal random variable having mean and variance . As a consequence, Slutsky's theorem applies and the product converges in distribution toBy elementary rules on linear transformations of normal random variables, this has a normal distribution with mean and variance

The next example shows how the delta method can be applied.

Example Suppose that a sequence is asymptotically normal with asymptotic mean and asymptotic variance , that is,We want to derive the asymptotic distribution of the sequence . The functionis continuously differentiable, so we can apply the delta method. The asymptotic mean of the transformed sequence isIn order to compute the asymptotic variance, we need to take the first derivative of the function , which isand evaluate it at :Therefore, the asymptotic variance isand we can write

The delta method generalizes also to multivariate settings, as stated by the following proposition.

Proposition Let be a sequence of random vectors such thatwhere is a multivariate normal distribution with mean and covariance matrix , is a constant vector, and indicates convergence in distribution. Let . If all the entries of have continuous partial derivatives with respect to , then where is the Jacobian of , i.e., the matrix of partial derivatives of the entries of with respect to the entries of .

The next example shows how the multivariate delta method can be applied.

Example Suppose that a sequence of random vectors satisfieswhere the asymptotic mean isand the asymptotic covariance matrix isDenote the two components of by and . We want to derive the asymptotic distribution of the sequence . The functionis continuously differentiable, so we can apply the delta method. The asymptotic mean of the transformed sequence isIn order to compute the asymptotic covariance matrix, we need to compute the Jacobian of the function , which isand evaluate it at :Therefore, the asymptotic variance isand we can write

Below you can find some exercises with explained solutions.

Let be an asymptotically normal sequence with asymptotic mean and asymptotic variance , that is,Derive the asymptotic distribution of the sequence .

Solution

The functionis continuously differentiable, so we can apply the delta method. The asymptotic mean of the transformed sequence isIn order to compute the asymptotic variance, we need to take the first derivative of the function , which isand evaluate it at :Therefore, the asymptotic variance isand we can write

Let be a sequence of random vectors satisfyingwhere the asymptotic mean isand the asymptotic covariance matrix isDenote the two entries of by and . Derive the asymptotic distribution of the sequence of products

Solution

We can apply the delta method because the functionis continuously differentiable. The asymptotic mean of the transformed sequence isThe Jacobian of the function isBy evaluating it atwe obtainTherefore, the asymptotic covariance matrix isand we can write

Let be a sequence of random vectors satisfyingwhere the asymptotic mean isand the asymptotic covariance matrix isDenote the two entries of by and . Derive the asymptotic distribution of the sequence of vectors where the two entries of satisfy

Solution

We can apply the delta method because the functionsare continuously differentiable. The asymptotic mean of the transformed sequence is a vector whose entries are The Jacobian of the function isBy evaluating it atwe obtainAs a consequence, the asymptotic covariance matrix isThus,where and have been calculated above.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Wald test
- Student t distribution
- Delta method
- Independent events
- Uniform distribution
- Conditional probability

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

Share