StatLect
Index > Glossary

Size of a test

In hypothesis testing, the size of a test is the (maximum) probability of committing a Type I error, that is, of incorrectly rejecting the null hypothesis when the null hypothesis is true.

Table of Contents

Definition

Suppose we are conducting a test about a parameter $	heta $ that can take any value in a parameter space $Theta $. The null hypothesis is that $	heta $ belongs to a given set [eq1]. Denote by [eq2] the power function of the test. The function [eq2] gives us the probability of rejecting the null hypothesis when the true parameter is $	heta $ (for any $	heta in Theta $). Given these assumptions, the size of the test can be defined as follows.

Definition The size of the test, denoted by $lpha $, is [eq4]

When the null hypothesis is simple, that is, the set $Theta _{R}$ contains only one parameter (denote it by $	heta _{0}$), the above definition becomes[eq5]

In other words, if the null hypothesis does not specify an exact value for the parameter, but a whole set of parameters, then we need to take the maximum of the power function over that set in order to compute the size; otherwise, if the null hypothesis specifies only one parameter, then it suffices to compute the value of the power function in correspondence of that parameter.

Example

For example, suppose we are testing the null hypothesis that the mean mu of a normal distribution is equal to $mu _{0}$. The variance of the distribution, denoted by sigma^2, is supposed to be known. We observe a sample of n independent draws [eq6] from the distribution and we compute the z-statistic[eq7]where Xbar_n is the sample mean: [eq8]

We select a critical value $z$ and reject the null hypothesis if [eq9]

It can be proved (see Hypothesis testing about the mean) that the power function of the test is[eq10]where $Fleft( x
ight) $ is the cumulative distribution function of a standard normal random variable.

The size of the test is[eq11]

By the symmetry of the standard normal distribution around 0$,$, we have that[eq12]

As a consequence, we can write the size of the test as[eq13]

In order to better understand this result, consider that under the null hypothesis the z-statistic has a standard normal distribution, that is, a normal distribution with mean equal to 0 and variance equal to 1. If you set, for example, $z=1.96,$, then you will reject the null in two cases:

But we know that if $Z_{n}$ has a standard normal distribution, then[eq14]

Thus the size of the test is 5%: [eq15]

The following plot shows the probability density function of the z-statistic. The black vertical segments indicate the two critical values. When the value of the z-statistic falls in one of the two tails of the distribution (which are separated from the center of the distribution by the two critical values), the null hypothesis is incorrectly rejected. The area under the probability density function in the two tails, colored with turquoise, is the probability of rejection, that is, the size of the test. The area under the probability density function in the center of the distribution, colored with lavender, is the probability of acceptance.

Probability density function of the z-statistic. The size of the test is the area in the two tails of the distribution.

How to adjust the size of a test

In the previous example the size of the test was 5%. What if we want to decrease the size to 1%? How can this be achieved?

In general, the size of a test can be modified by changing the critical value(s) of the test, that is, by reducing or increasing the size (and hence the probability) of the critical region (remember that the critical region is the set of values of the test statistic that lead to rejection of the null hypothesis).

In the previous example, we can increase the size of the test by increasing the critical value $z$. In particular, note that [eq13]implies[eq17]So, if our desired size is $lpha =0.01$, then we need to search for what value the cumulative distribution function of the normal distribution is equal to $lpha /2=0.005$. By using normal distribution tables or a computer, we find that the desired value is $-2.576$. As a consequence, the new critical value is [eq18]

More details

More details about the concept of size of a test can be found in the lecture entitled Hypothesis testing.

Keep reading the glossary

Previous entry: Sample variance

Next entry: Standard deviation

The book

Most of the learning materials found on this website are now available in a traditional textbook format.