Search for probability and statistics terms on Statlect

Score test

by , PhD

The score test, also known as Lagrange Multiplier (LM) test, is a hypothesis test used to check whether some parameter restrictions are violated.

A score test can be performed after estimating the parameters by maximum likelihood (ML).

Table of Contents

The null hypothesis

The score test is used to deal with null hypotheses of the following kind:[eq1]where:

Example of null hypothesis

As explained in this introductory lecture, all the most common null hypotheses and parameter restrictions can be written in the form [eq4].

Example If $	heta _{0}$ has two entries $	heta _{0,1}$ and $	heta _{0,2}$, and the null hypothesis is [eq5], then[eq6]There are $p=2$ parameters and $r=1$ tested restrictions.

Restricted maximum likelihood

The score test is based on the solution of the constrained maximum likelihood problem[eq7]where:

Thus, the parameter estimate [eq10] satisfies all the tested restrictions.

The score statistic

The test statistic, called score statistic (or Lagrange Multiplier statistic), is[eq11]where:

Another way to write the score statistic

A popular estimator of the asymptotic covariance matrix is the so-called Hessian estimator:[eq15]where[eq16]is the Hessian (i.e., the matrix of second partial derivatives of the log-likelihood with respect to the parameters).

If we plug this estimator in the above formula for the score statistic, we obtain:[eq17]

Many sources report this formula, but bear in mind that it is only a particular implementation of the LM test. If we use different estimators of the asymptotic covariance matrix, we obtain different formulae.

Technical conditions

In order to derive the asymptotic properties of the statistic $LM_{n}$, the following assumptions will be maintained:

Asymptotic distribution of the test statistic

The Lagrange Multiplier statistic converges to a Chi-square distribution.

Proposition Provided that some technical conditions are satisfied (see above), and provided that the null hypothesis [eq20] is true, the statistic $LM_{n}$ converges in distribution to a Chi-square distribution with $r$ degrees of freedom.


Denote by [eq21] the unconstrained maximum likelihood estimate:[eq22]By the Mean Value Theorem, we have that[eq23]where [eq24] is an intermediate point (a vector whose components are strictly comprised between the components of [eq25] and those of [eq26]). Since [eq27] $Theta _{R}$, we have that[eq28]Therefore,[eq29]Again by the Mean Value Theorem, we have that[eq30]where [eq31] is the Hessian matrix (a matrix of second partial derivatives) and [eq32] is an intermediate point (actually, to be precise, there is a different intermediate point for each row of the Hessian). Because the gradient is zero at an unconstrained maximum, we have that[eq33]and, as a consequence,[eq34]and [eq35]It descends that[eq36]Now, [eq37]where $lambda $ is a $r	imes 1$ vector of Lagrange multipliers. Thus, we have that[eq38]Solving for $lambda $, we obtain[eq39]Now, the score statistic can be written as[eq40]Plugging in the previously derived expression for $lambda ,$, the statistic becomes[eq41]where[eq42]Given that under the null hypothesis both [eq43] and [eq44] converge in probability to $	heta _{0}$, also [eq45] and [eq46] converge in probability to $	heta _{0}$, because the entries of [eq47] and [eq48] are strictly comprised between the entries of [eq49] and [eq50]. Moreover,[eq51]where V is the asymptotic covariance matrix of [eq52]. We had previously assumed that also $widehat{V}_{n}$ converges in probability to V. Therefore, by the continuous mapping theorem, we have the following results[eq53]By putting together everything we have derived so far, we can write the score statistic as a sequence of quadratic forms [eq54]where[eq55]and [eq56]But in the lecture on the Wald test, we have proved that such a sequence converges in distribution to a Chi-square random variable with a number of degrees of freedom equal to [eq57].

The test

In the score test, the null hypothesis is rejected if the score statistic exceeds a pre-determined critical value $z$, that is, if[eq58]

The size of the test can be approximated by its asymptotic value[eq59]where $Fleft( z
ight) $ is the distribution function of a Chi-square random variable with $r$ degrees of freedom.

We can choose $z$ so as to achieve a pre-determined size, as follows:[eq60]


Here is an example of how to perform a Lagrange Multiplier test.

The parameter space

Let the parameter space be the set of all $2$-dimensional vectors, that is, [eq61].

Denote the entries of the true parameter $	heta _{0}$ by $	heta _{0,1}$ and $	heta _{0,2}$.

The restriction

Suppose that we want to test the restriction[eq62]

In this case, the function [eq63] is a function [eq64] defined by[eq65]

The Jacobian

We have that $r=1$ and the Jacobian of $g$ is[eq66]whose rank is equal to $r=1$.

Note that the Jacobian does not depend on $	heta $.

The restricted estimate

We then maximize the log-likelihood function with respect to $	heta _{2}$ (keeping $	heta _{1}$ fixed at $	heta _{1}=0$).

Suppose that we obtain the following estimates of the parameter and of the asymptotic covariance matrix:[eq67]where $70$ is the sample size.

Suppose also that the value of the score is[eq68]

The LM statistic

Then, the score statistic is [eq69]

The statistic has a Chi-square distribution with $r=1$ degrees of freedom.

The critical value

Suppose that we want the size of our test to be $lpha =1%$.

Then, the critical value $z$ is[eq70]where $Fleft( z
ight) $ is the cumulative distribution function of a Chi-square random variable with 1 degree of freedom.

The value of [eq71] can be calculated with any statistical software (we did it in MATLAB, using the command chi2inv(0.99,1)).

The decision

Thus, the test statistic exceeds the critical value[eq72]and we reject the null hypothesis.

How to cite

Please cite as:

Taboga, Marco (2021). "Score test", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.