StatlectThe Digital Textbook
Index > Fundamentals of statistics

Set estimation

In statistical inference, a sample is employed to make statements about the probability distribution from which the sample has been generated (see the lecture entitled Statistical inference). The sample $xi $ can be regarded as a realization of a random vector $Xi $, whose joint distribution function, denoted by [eq1], is unknown, but is assumed to belong to a set of distribution functions $Phi $, called statistical model.

In a parametric model, the set $Phi $ is put into correspondence with a set [eq2] of p-dimensional real vectors. $Theta $ is called the parameter space and its elements are called parameters. Denote by $	heta _{0}$ the true parameter, that is, the parameter that is associated with the unknown distribution function [eq3] from which the sample $xi $ was actually generated. For concreteness, $	heta _{0}$ is assumed to be unique. This lecture discusses a kind of inference about $	heta _{0}$ called set estimation.

Confidence sets

Roughly speaking, set estimation is the act of choosing a subset $T$ of the parameter space ($Tsubseteq Theta $) in such a way that $T$ has a high probability of containing the true (and unknown) parameter $	heta _{0}$. The chosen subset $T$ is called a set estimate of $	heta _{0}$ or a confidence set for $	heta _{0}$.

When the parameter space $Theta $ is a subset of the set of real numbers R and the subset $T$ is chosen among the intervals of R (e.g. intervals of the kind $left[ a,b
ight] $), we speak about interval estimation (instead of set estimation), interval estimate (instead of set estimate) and confidence interval (instead of confidence set).

When the set estimate $T$ is produced using a predefined rule (a function) that associates a set estimate $T$ to each $xi $ in the support of $Xi $, we can write[eq4]

The function [eq5] is called a set estimator. Often, the symbol $T$ is used to denote both the set estimate and the set estimator. The meaning is usually clear from the context.

Coverage probabilities and confidence coefficients

As we already said, set estimation is the act of choosing a subset $T$ of the parameter space in such a way that $T$ has a high probability of containing the true parameter $	heta _{0}$. The probability that $T$ contains the true parameter is called coverage probability and it is usually chosen by the statistician. Intuitively, before observing the data the statistician makes a statement: [eq6]where $Theta $ is the parameter space, containing all the parameters that are deemed plausible. The statistician believes the statement to be true, but the statement is not very informative, because $Theta $ is a very large set. After observing the data, she makes a more informative statement:[eq7]This statement is more informative, because $T$ is smaller than $Theta $, but it has a positive probability of being wrong (which is the complement to 1 of the coverage probability). In controlling this probability, the statistician faces a trade-off: if she decreases the probability of being wrong, then her statements become less informative; on the contrary, if she increases the probability of being wrong, then her statements become more informative.

In formal terms, the coverage probability of a set estimator is defined as follows:[eq8]where the notation [eq9] is used to indicate the fact that the probability is calculated using the distribution function [eq10] associated to the true parameter $	heta _{0}$. It is important to note that in the above definition of coverage probability the random quantity is the interval [eq11], while the parameter is fixed.

In practice, the coverage probability is seldom known, because it depends on the unknown parameter $	heta _{0}$ (although in some cases it is equal for all parameters belonging to the parameter space). When the coverage probability is not known, it is customary to compute the confidence coefficient [eq12], which is defined as follows:[eq13]In other words, the confidence coefficient [eq14] is equal to the smallest possible coverage probability. The confidence coefficient is also often called level of confidence.

Size of a confidence set

We already mentioned that there is a trade-off in the construction and choice of a set estimator. On the one hand, we want our set estimator $T$ to have a high coverage probability, that is, we want the set $T$ to include the true parameter with a high probability. On the other hand, we want the size of $T$ to be as small as possible, so as to make our interval estimate more precise. What do we mean by size of $T$? If the parameter space $Theta $ is unidimensional and $T$ is an interval estimate, then the size of $T$ is just its length. If the space $Theta $ is multidimensional, then the size of $T$ is its volume. The size of a confidence set is also called measure of a confidence set (for those who have a grasp of measure theory, the name stems from the fact that Lebesgue measure is the generalization of volume in multidimensional spaces). If we denote by [eq15] the size of a confidence set, then we can also define the expected size of a set estimator $T$:[eq16]where the notation [eq17] is used to indicate the fact that the expected value is calculated using the distribution function [eq18] associated to the true parameter $	heta _{0}$. Like the coverage probability, also the expected size of a set estimator depends on the unknown parameter $	heta _{0}$. Hence, unless it is a constant function of $	heta _{0}$, one needs to somehow estimate it or to take the infimum over all possible values of the parameter, as we did above for coverage probabilities.

Other criteria to evaluate set estimators

Although size is probably the simplest criterion to evaluate and select set estimators, there are several other criteria. We do not discuss them here, but we refer the reader to the very nice exposition in Berger, R. L. and G. Casella (2002) "Statistical inference", Duxbury Advanced Series.

Examples

Examples of set estimation problems can be found in the following lectures:

  1. Set estimation of the mean (examples of set estimation of the mean of an unknown distribution);

  2. Set estimation of the variance (examples of set estimation of the variance of an unknown distribution).

The book

Most learning materials found on this website are now available in a traditional textbook format.