Index > Fundamentals of statistics > Maximum likelihood - Hypothesis testing

Score test

The score test, also known as Lagrange Multiplier (LM) test, is a hypothesis test used to check whether some parameter restrictions are violated.

A score test can be performed after estimating the parameters by maximum likelihood (ML).

Table of contents

The null hypothesis
Example of null hypothesis
Restricted maximum likelihood
The score statistic
Another way to write the score statistic
Technical conditions
Asymptotic distribution of the test statistic
The test
Example

The null hypothesis

The score test is used to deal with null hypotheses of the following kind:where:

$heta _{0}$ is an unknown parameter belonging to a parameter space ;
is a vector-valued function ();
is the number of parameters;
is the number of restrictions being tested.

Example of null hypothesis

As explained in this introductory lecture, all the most common null hypotheses and parameter restrictions can be written in the form .

Example If $heta _{0}$ has two entries $heta _{0,1}$ and $heta _{0,2}$ , and the null hypothesis is , thenThere are parameters and tested restrictions.

Restricted maximum likelihood

The score test is based on the solution of the constrained maximum likelihood problemwhere:

the set contains all the parameters that satisfy the tested restriction;
$xi _{n}$ is the sample of observed data;
is the sample size;
is the likelihood function.

Thus, the parameter estimate satisfies all the tested restrictions.

The score statistic

The test statistic, called score statistic (or Lagrange Multiplier statistic), iswhere:

the column vector is the gradient of the log-likelihood function (called score); in other words, is the vector of partial derivatives of the log-likelihood function with respect to the entries of the parameter vector ;
the matrix $widehat{V}_{n}$ is a consistent estimate of the asymptotic covariance matrix of the estimator (see Maximum likelihood - Covariance matrix estimation).

Another way to write the score statistic

A popular estimator of the asymptotic covariance matrix is the so-called Hessian estimator:whereis the Hessian (i.e., the matrix of second partial derivatives of the log-likelihood with respect to the parameters).

If we plug this estimator in the above formula for the score statistic, we obtain:

Many sources report this formula, but bear in mind that it is only a particular implementation of the LM test. If we use different estimators of the asymptotic covariance matrix, we obtain different formulae.

Technical conditions

In order to derive the asymptotic properties of the statistic $LM_{n}$ , the following assumptions will be maintained:

the sample and the likelihood function satisfy some set of conditions that are sufficient to guarantee the consistency and asymptotic normality of (see the lecture on maximum likelihood estimation for a set of such conditions);
for each , the entries of are continuously differentiable with respect to all the entries of ;
the matrix of the partial derivatives of the entries of with respect to the entries of , called the Jacobian of and denoted by , has rank .

Asymptotic distribution of the test statistic

The Lagrange Multiplier statistic converges to a Chi-square distribution.

Proposition Provided that some technical conditions are satisfied (see above), and provided that the null hypothesis is true, the statistic $LM_{n}$ converges in distribution to a Chi-square distribution with degrees of freedom.

Proof

Denote by the unconstrained maximum likelihood estimate:By the Mean Value Theorem, we have thatwhere is an intermediate point (a vector whose components are strictly comprised between the components of and those of ). Since $Theta _{R}$ , we have thatTherefore,Again by the Mean Value Theorem, we have thatwhere is the Hessian matrix (a matrix of second partial derivatives) and is an intermediate point (actually, to be precise, there is a different intermediate point for each row of the Hessian). Because the gradient is zero at an unconstrained maximum, we have thatand, as a consequence,and It descends that [eq36] Now, where is a vector of Lagrange multipliers. Thus, we have that [eq38] Solving for , we obtain [eq39] Now, the score statistic can be written as [eq40] Plugging in the previously derived expression for , the statistic becomeswhere [eq42] Given that under the null hypothesis both and converge in probability to $heta _{0}$ , also and converge in probability to $heta _{0}$ , because the entries of and are strictly comprised between the entries of and . Moreover,where is the asymptotic covariance matrix of . We had previously assumed that also $widehat{V}_{n}$ converges in probability to . Therefore, by the continuous mapping theorem, we have the following results [eq53] By putting together everything we have derived so far, we can write the score statistic as a sequence of quadratic forms whereand But in the lecture on the Wald test, we have proved that such a sequence converges in distribution to a Chi-square random variable with a number of degrees of freedom equal to .

The test

In the score test, the null hypothesis is rejected if the score statistic exceeds a pre-determined critical value , that is, if

The size of the test can be approximated by its asymptotic valuewhere is the distribution function of a Chi-square random variable with degrees of freedom.

We can choose so as to achieve a pre-determined size, as follows:

Example

Here is an example of how to perform a Lagrange Multiplier test.

The parameter space

Let the parameter space be the set of all -dimensional vectors, that is, .

Denote the entries of the true parameter $heta _{0}$ by $heta _{0,1}$ and $heta _{0,2}$ .

The restriction

Suppose that we want to test the restriction

In this case, the function is a function defined by

The Jacobian

We have that and the Jacobian of is [eq66] whose rank is equal to .

Note that the Jacobian does not depend on .

The restricted estimate

We then maximize the log-likelihood function with respect to $heta _{2}$ (keeping $heta _{1}$ fixed at $heta _{1}=0$ ).

Suppose that we obtain the following estimates of the parameter and of the asymptotic covariance matrix: [eq67] where is the sample size.

Suppose also that the value of the score is [eq68]

The LM statistic

Then, the score statistic is [eq69]

The statistic has a Chi-square distribution with degrees of freedom.

The critical value

Suppose that we want the size of our test to be .

Then, the critical value iswhere is the cumulative distribution function of a Chi-square random variable with degree of freedom.

The value of can be calculated with any statistical software (we did it in MATLAB, using the command chi2inv(0.99,1)).

The decision

Thus, the test statistic exceeds the critical valueand we reject the null hypothesis.

How to cite

Please cite as:

Taboga, Marco (2021). "Score test", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-statistics/score-test.