Search for probability and statistics terms on Statlect
StatLect

Wald test

by , PhD

The Wald test is a test of hypothesis usually performed on parameters that have been estimated by maximum likelihood (ML).

Table of Contents

The null hypothesis

We assume that an unknown p-dimensional parameter vector $	heta _{0}$ has been estimated by ML.

We want to test the null hypothesis that $r$ equations (possibly nonlinear) are satisfied:[eq1]where [eq2] is a vector valued function, with $rleq p$.

The lecture on hypothesis testing in maximum likelihood framework explains that the most common null hypotheses can be written in this form.

Assumptions

Let the parameter space be [eq3].

We assume that the following technical conditions are satisfied:

The ML estimator

Let [eq5] be the estimate of the $p	imes 1$ parameter $	heta _{0}$ obtained by maximizing the log-likelihood over the whole parameter space $Theta $:[eq6]where [eq7] is the likelihood function and $xi _{n}$ is the sample.

We assume that the sample and the likelihood function satisfy some set of conditions that are sufficient to guarantee the consistency and asymptotic normality of [eq8] (see the lecture on maximum likelihood for a set of such conditions).

The Wald statistic

Here is the formula for the test statistic used in the Wald test:[eq9]where n is the sample size, and $widehat{V}_{n}$ is a consistent estimate of the asymptotic covariance matrix of [eq10] (see Covariance matrix of the MLE).

The asymptotic distribution of the test statistic

Asymptotically, the test statistic has a Chi-square distribution.

Proposition Under the null hypothesis that [eq11], the Wald statistic $W_{n}$ converges in distribution to a Chi-square distribution with $r$ degrees of freedom.

Proof

We have assumed that [eq12] is consistent and asymptotically normal, which implies that [eq13] converges in distribution to a multivariate normal random variable with mean $	heta _{0}$ and asymptotic covariance matrix V, that is,[eq14]Now, by the delta method, we have that[eq15]But [eq16], so that[eq17]We have assumed that [eq18] and $widehat{V}_{n}$ are consistent estimators, that is,[eq19]where [eq20] denotes convergence in probability. Therefore, by the continuous mapping theorem, we have that[eq21]Thus we can write the Wald statistic as a sequence of quadratic forms [eq22]where[eq23]converges in distribution to a normal random vector $G$ with mean zero, and [eq24]converges in probability to [eq25]. By a standard result (see Exercise 2 in the lecture on Slutsky's theorem), such a sequence of quadratic forms converges in distribution to a Chi-square random variable with a number of degrees of freedom equal to [eq26].

The test

In the Wald test, the null hypothesis is rejected if[eq27]where $z$ is a pre-determined critical value.

The size of the test can be approximated by its asymptotic value[eq28]where $Fleft( z
ight) $ is the distribution function of a Chi-square random variable with $r$ degrees of freedom.

The critical value $z$ is chosen so as to achieve a pre-determined size, as follows:[eq29]

Example

This example shows how to use the Wald test to test a simple linear restriction.

Let the parameter space be the set of all $2$-dimensional vectors: [eq30]

The estimates

Suppose that we have obtained the following estimates of the parameter and of the asymptotic covariance matrix:[eq31]where $90$ is the sample size.

The null hypothesis

We want to test the restriction [eq32]where $	heta _{0,1}$ and $	heta _{0,2}$ denote the first and second entries of $	heta _{0}$.

Then, the function [eq33] is a function [eq34] defined by[eq35]

In this case, $r=1$.

The Jacobian

The Jacobian of $g$ is[eq36]which has rank $r=1$.

Note also that it does not depend on $	heta $.

The test statistic

We have[eq37]

We can substitute these values in the formula for the Wald statistic:[eq38]

Our test statistic has a Chi-square distribution with $r=1$ degrees of freedom.

The critical value

Suppose that we want our test to have size $lpha =5%$.

Then, our critical value $z$ is[eq39]where $Fleft( z
ight) $ is the distribution function of a Chi-square random variable with 1 degree of freedom.

The value of [eq40] can be calculated with any statistical software (for, example, in MATLAB with the command chi2inv(0.95,1)).

The decision

Therefore, the test statistic does not exceed the critical value[eq41]and we do not reject the null hypothesis.

How to cite

Please cite as:

Taboga, Marco (2021). "Wald test", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-statistics/Wald-test.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.