Index > Glossary


by Marco Taboga, PhD

In the theory of parameter estimation, an estimator is a function that associates a parameter estimate to each possible sample we can observe.

Table of Contents


In a parameter estimation problem, we need to choose a parameter [eq1]from a parameter space $\Theta $, by using a sample $\xi $ (a set of observations from an unknown probability distribution).

The parameter $\widehat{\theta }$ is our best guess of the true and unknown parameter $\theta _{0}$, which is associated to (or describes) the probability distribution that generated the sample. The parameter $\widehat{\theta }$ is called an estimate of $\theta _{0}$.

When the estimate $\widehat{\theta }$ is produced by using a predefined rule that associates a parameter estimate $\widehat{\theta }$ to each possible sample $\xi $, we can write $\widehat{\theta }$ as a function of $\xi $:[eq2]

The function [eq3] is called an estimator.

Estimators as random variables

The sample $\xi $, before being observed, is regarded as a random variable, drawn from the distribution of interest. Therefore, the estimator $\widehat{\theta }$, being a function of $\xi $, is also regarded as a random variable.

After the sample $\xi $ is observed, the realization [eq3] of the estimator is called an estimate of the true parameter $\theta _{0}$. In other words, an estimate is a realization of an estimator.


Commonly found examples of estimators are:

How estimators are assessed

Different estimators of the same parameter are often compared by looking at their mean squared error, which is equal to the expected value of the squared difference between the estimator and the true value of the parameter. For an example of such comparisons, see the lecture on Ridge estimation.

More details

More details about estimators can be found in the lecture entitled Point estimation, which discusses the concept of estimator and the main criteria used to evaluate estimators.

Keep reading the glossary

Previous entry: Estimate

Next entry: Event

The book

Most of the learning materials found on this website are now available in a traditional textbook format.