Loss function
In the theory of point estimation, a loss function quantifies the losses associated to the errors committed while estimating a parameter. Often the expected value of the loss, called statistical risk, is used to compare two or more estimators: in such comparisons, the estimator having the least expected loss is usually deemed preferable.
Table of contents
Definition
The following is a possible definition.
Definition
Let
be an unknown parameter and
an estimate of
.
The estimation error is the
difference![[eq1]](/images/loss_function__4.png)
The
loss function is a function mapping estimation errors to the set of real
numbers.
Examples
Let
![[eq2]](/images/loss_function__5.png)
denote
the Euclidean norm. Commonly used loss functions are:
-
the absolute estimation error
![[eq3]](/images/loss_function__6.png)
which
coincides with the absolute value of the error
![[eq4]](/images/loss_function__7.png)
when the parameter is a scalar;
-
the squared estimation error
![[eq5]](/images/loss_function__8.png)
which
coincides with the the square of the error
![[eq6]](/images/loss_function__9.png)
when the parameter is a scalar.
In both cases, the larger the estimation error is, the larger is the loss. The expected value of the former is called mean absolute error (MAE), while the expectation of the latter is known as mean squared error (MSE).
More details
Loss functions, estimation errors and statistical risk are explained in more detailed in the lecture entitled Point estimation.
Keep reading the glossary
Previous entry: Joint probability mass function
Next entry: Marginal distribution function