In linear regression analysis, an estimator of the asymptotic covariance matrix of the OLS estimator is said to be heteroskedasticity-robust if it converges asymptotically to the true value even when the variance of the errors of the regression is not constant.
In this case, also the standard errors, which are equal to the square roots of the diagonal entries of the covariance matrix, are said to be heteroskedasticity-robust.
Table of contents
Consider the linear regression modelwhere:
is the dependent variable;
is the vector of regressors;
is the vector of regression coefficients;
is the zero-mean error term.
There are observations in the sample:
The ordinary least squares (OLS) estimator of can be written as
Under appropriate conditions, the OLS estimator is asymptotically normal:where:
denotes convergence in distribution;
is the asymptotic covariance matrix of the OLS estimator;
denotes a multivariate normal distribution with mean vector equal to and covariance matrix equal to .
The standard errors are the estimates of the standard deviations of the entries of .
Denote by an estimator of .
Then, the covariance matrix of is approximated byand the standard errors are equal to the square roots of the diagonal entries of the latter matrix.
The errors of the regression are said to be conditionally homoskedastic if their variance is constant:where is a constant.
If the conditional variance is not constant, the errors are said to be conditionally heteroskedastic, and the regression is said to be affected by heteroskedasticity.
An estimator of the asymptotic covariance matrix is heteroskedasticity-robust if it is consistent even when the errors are conditionally heteroskedastic.
Consistent means thatwhere denotes convergence in probability.
Under mild technical conditions, the asymptotic covariance matrix iswhereis the so-called long-run covariance matrix.
The matrix is consistently estimated by
Therefore, by the Continuous Mapping Theorem, if we can find a consistent estimator of , then the asymptotic covariance matrix is consistently estimated by
Suppose that the following assumptions about the sequence hold:
no serial correlation:if ;
weak stationarity: does not depend on
Then, the long-run covariance matrix can be written as
The proof is as follows:
Note that the zero-mean assumption is the same as the orthogonality assumption usually needed to prove the consistency of the OLS estimator.
Under mild technical conditions, the long-run covariance matrix is consistently estimated bywhere the residuals are defined as
The sample averageconverges in probability to if the sequence satisfies the conditions of a Law of Large Numbers (the mild technical conditions mentioned above). The errors in the last formula can be replaced by the residuals , as the latter converge in probability to the former when the sample size increases (a formal proof can be found here).
If we plug the formula for in the expression we had previously derived for the estimator of the asymptotic covariance matrix, we obtain:
This estimator is robust to heteroskedasticity.
As a matter of fact, we did not assume homoskedasticity to prove its consistency.
The square roots of the diagonal entries of the matrixare known as heteroskedasticity-robust standard errors.
Using matrix notation, we can write the expression above in a more compact form.
Define the vectors and matrices
Then, the heteroskedasticity-robust covariance matrix is
Compare the formulae above with those for the non-robust estimatorwhere
This estimator is non-robust to heteroskedasticity.
In fact, in order to prove its consistency, we need to assume conditional homoskedasticity for every with constant.
Under the hypothesis of homoskedasticity, we can write the long-run covariance matrix as follows:which is consistently estimated byThe estimator of the asymptotic covariance matrix becomes:Hence, the estimator of the covariance matrix is
Heteroskedasticity-robust standard errors go by many different names:
heteroskedasticity-consistent standard errors;
Eicker-Huber-White standard errors;
Huber-White standard errors;
White standard errors.
More mathematical details and proofs of the facts stated above can be found in the lecture on the properties of the OLS estimator.
Previous entry: Realization of a random variable
Next entry: Sample mean
Please cite as:
Taboga, Marco (2021). "Hetroskedasticity-robust standard errors", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/glossary/heteroskedasticity-robust-standard-errors.
Most of the learning materials found on this website are now available in a traditional textbook format.