Search for probability and statistics terms on Statlect
StatLect

Size of a test

by , PhD

In hypothesis testing, the size of a test is the (maximum) probability of committing a Type I error, that is, of incorrectly rejecting the null hypothesis when it is true.

Table of Contents

Null hypothesis

Suppose that we are conducting a test about a parameter $	heta $ that can take any value in a parameter space $Theta $.

The null hypothesis $H_{0}$ is that $	heta $ belongs to a given set [eq1]:[eq2]

Power function

Denote by [eq3] the power function of the test.

For any $	heta in Theta $, the function [eq3] gives us the probability of rejecting the null hypothesis when the true parameter is $	heta $:[eq5]

Note that [eq3] depends on:

Definition

The size of the test can be defined as follows.

Definition The size of the test, denoted by $lpha $, is [eq7]

Thus, we consider all the cases in which the null is true ([eq8]).

For each case, we compute the probability of (incorrect) rejection.

The size is equal to the largest value we find (worst-case scenario).

Simple vs composite hypothesis

When the null hypothesis is simple, that is, the set $Theta _{R}$ contains only one parameter (denote it by $	heta _{0}$), the above definition becomes[eq9]

Thus, we have two cases:

  1. if the null hypothesis does not specify an exact value for the parameter, but a whole set of parameters, then we need to take the maximum of the power function over that set in order to compute the size;

  2. otherwise, if the null hypothesis specifies only one parameter, then it suffices to compute the value of the power function corresponding to that parameter.

Size vs level of significance

Many authors use the term level of significance as a synonym for size.

For many others, the level of significance is an upper bound to the size, that is, a constant $overline{lpha }$ that satisfies[eq10]

Example - Size of a test for the mean

For example, suppose that we are testing the null hypothesis that the mean mu of a normal distribution is equal to $mu _{0}$.

The variance of the distribution, denoted by sigma^2, is supposed to be known.

How the test is conducted

We observe a sample of n independent draws [eq11] from the distribution and we compute the z-statistic[eq12]where Xbar_n is the sample mean: [eq13]

We select a critical value $z$ and reject the null hypothesis if [eq14]

How the size is derived

It can be proved (see Hypothesis testing about the mean) that the power function of the test is[eq15]where $Fleft( x
ight) $ is the cumulative distribution function of a standard normal random variable.

The size of the test is[eq16]

By the symmetry of the standard normal distribution around 0$,$, we have that[eq17]

As a consequence, we can write the size of the test as[eq18]

How to visualize the size

In order to better understand this result, consider that under the null hypothesis the z-statistic has a standard normal distribution, that is, a normal distribution with mean equal to 0 and variance equal to 1.

If you set, for example, $z=1.96,$, then you will reject the null in two cases:

But we know that if $Z_{n}$ has a standard normal distribution, then[eq19]

Thus the size of the test is 5%: [eq20]

The following plot shows the probability density function of the z-statistic.

Probability density function of the z-statistic. The size of the test is the area in the two tails of the distribution.

The black vertical segments indicate the two critical values.

When the value of the z-statistic falls in one of the two tails of the distribution (which are separated from the center of the distribution by the two critical values), the null hypothesis is incorrectly rejected.

The area under the probability density function in the two tails, colored with turquoise, is the probability of rejection, that is, the size of the test.

The area under the probability density function in the center of the distribution, colored with lavender, is the probability of acceptance.

How to adjust the size of a test

In the previous example the size of the test was 5%. What if we want to decrease the size to 1%? How can this be achieved?

In general, the size of a test can be modified by changing the critical value(s) of the test, that is, by reducing or increasing the size (and hence the probability) of the critical region (remember that the critical region is the set of values of the test statistic that lead to rejection of the null hypothesis).

In the previous example, we can decrease the size of the test by increasing the critical value $z$. In particular, note that [eq18]implies[eq22]

So, if our desired size is $lpha =0.01$, then we need to search for what value the cumulative distribution function of the normal distribution is equal to $lpha /2=0.005$.

By using normal distribution tables or a computer, we find that the desired value is $-2.576$.

As a consequence, the new critical value is [eq23]

More details

More details about the concept of size of a test can be found in the lecture entitled Hypothesis testing.

Keep reading the glossary

Previous entry: Score vector

Next entry: Standard deviation

How to cite

Please cite as:

Taboga, Marco (2021). "Size of a test", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/glossary/size-of-a-test.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.