The Gamma distribution can be thought of as a generalization of the Chi-square distribution.
If a random variable has a Chi-square distribution with degrees of freedom and is a strictly positive constant, then the random variable defined as has a Gamma distribution with parameters and .
Gamma random variables are characterized as follows.
Definition Let be a continuous random variable. Let its support be the set of positive real numbers:Let . We say that has a Gamma distribution with parameters and if and only if its probability density function iswhere is a constant:and is the Gamma function.
A random variable having a Gamma distribution is also called a Gamma random variable.
To better understand the Gamma distribution, you can have a look at its density plots.
The expected value of a Gamma random variable is
It can be derived as follows:
The variance of a Gamma random variable is
It can be derived thanks to the usual variance formula ():
The moment generating function of a Gamma random variable is defined for any :
By using the definition of moment generating function, we obtainwhere the integral equals because it is the integral of the probability density function of a Gamma random variable with parameters and . Thus,Of course, the above integrals converge only if , i.e. only if . Therefore, the moment generating function of a Gamma random variable exists for all .
The characteristic function of a Gamma random variable is
It can be derived by using the definition of characteristic function and a Taylor series expansion:
The distribution function of a Gamma random variable iswhere the functionis called lower incomplete Gamma function and is usually evaluated using specialized computer algorithms.
This is proved as follows:
In the following subsections you can find more details about the Gamma distribution.
If a variable has the Gamma distribution with parameters and , thenwhere has a Chi-square distribution with degrees of freedom.
This can be easily proved using the formula for the density of a function of a continuous variable ( is a strictly increasing function of , since is strictly positive):The density function of a Chi-square random variable with degrees of freedom iswhere Therefore,which is the density of a Gamma distribution with parameters and .
Thus, the Chi-square distribution is a special case of the Gamma distribution because, when , we have
In other words, a Gamma distribution with parameters and is just a Chi square distribution with degrees of freedom.
By multiplying a Gamma random variable by a strictly positive constant, one obtains another Gamma random variable. If is a Gamma random variable with parameters and , then the random variable defined ashas a Gamma distribution with parameters and .
This can be easily seen using the result from the previous subsection:where has a Chi-square distribution with degrees of freedom. Therefore,In other words, is equal to a Chi-square random variable with degrees of freedom, divided by and multiplied by . Therefore, it has a Gamma distribution with parameters and .
In the lecture entitled Chi-square distribution we have explained that a Chi-square random variable with degrees of freedom ( integer) can be written as a sum of squares of independent normal random variables , ..., having mean and variance :
In the previous subsections we have seen that a variable having a Gamma distribution with parameters and can be written aswhere has a Chi-square distribution with degrees of freedom.
Putting these two things together, we obtainwhere we have definedBut the variables are normal random variables with mean and variance . Therefore, a Gamma random variable with parameters and can be seen as a sum of squares of independent normal random variables having mean and variance .
This page collects some plots of the Gamma distribution. These plots help us to understand how the shape of the Gamma distribution changes when its parameters are changed.
The following plot contains the graphs of two Gamma probability density functions.
the first graph (red line) is the probability density function of a Gamma random variable with degrees of freedom and mean ;
the second graph (blue line) is the probability density function of a Gamma random variable with degrees of freedom and mean .
Because in both cases, the two distributions have the same mean. However, by increasing the number of degrees of freedom from to , the shape of the distribution changes (the more the degrees of freedom are increased the more the distribution resembles a normal distribution).
The thin vertical lines indicate the means of the two distributions.
The following plot contains the graphs of two Gamma probability density functions:
the first graph (red line) is the probability density function of a Gamma random variable with degrees of freedom and mean ;
the second graph (blue line) is the probability density function of a Gamma random variable with degrees of freedom and mean .
Increasing the parameter changes the mean of the distribution from to . However, the two distributions have the same number of degrees of freedom (). Therefore, they have the same shape (one is the "stretched version of the other" - it would look exactly the same on a different scale).
Below you can find some exercises with explained solutions.
Let and be two independent Chi-square random variables having and degrees of freedom respectively. Consider the following random variables:What distribution do they have?
Being multiples of Chi-square random variables, the variables , and all have a Gamma distribution. The random variable has degrees of freedom and the random variable can be written aswhere . Therefore has a Gamma distribution with parameters and . The random variable has degrees of freedom and the random variable can be written aswhere . Therefore has a Gamma distribution with parameters and . The random variable has a Chi-square distribution with degrees of freedom, because and are independent (see the lecture entitled Chi-square distribution), and the random variable can be written aswhere . Therefore has a Gamma distribution with parameters and .
Let be a random variable having a Gamma distribution with parameters and . Define the following random variables:
What distribution do these variables have?
Multiplying a Gamma random variable by a strictly positive constant one still obtains a Gamma random variable. In particular, the random variable is a Gamma random variable with parameters and The random variable is a Gamma random variable with parameters and The random variable is a Gamma random variable with parameters and The random variable is also a Chi-square random variable with degrees of freedom (remember that a Gamma random variable with parameters and is also a Chi-square random variable when ).
Let , and be mutually independent normal random variables having mean and variance . Consider the random variableWhat distribution does have?
The random variable can be written as where , and are mutually independent standard normal random variables. The sum has a Chi-square distribution with degrees of freedom (see the lecture entitled Chi-square distribution). Therefore has a Gamma distribution with parameters and .
Most of the learning materials found on this website are now available in a traditional textbook format.