Search for probability and statistics terms on Statlect
StatLect
Index > Probability distributions > Normal distribution

Linear combinations of normal random variables

by , PhD

A property that makes the normal distribution very tractable from an analytical viewpoint is its closure under linear combinations:

The following sections present a generalization of this elementary property and then discuss some special cases, summarized in the following infographic.

Infographic summarizing all the results about linear combinations of normal random variables which are proved below.

Table of Contents

Linear transformation of a multivariate normal random vector

A linear transformation of a multivariate normal random vector also has a multivariate normal distribution.

Proposition Let X be a Kx1 multivariate normal random vector with mean mu and covariance matrix V. Let A be an $L imes 1$ real vector and $B$ an $L imes K$ full-rank real matrix. Then, the $L imes 1$ random vector Y defined by[eq1]has a multivariate normal distribution with mean[eq2]and covariance matrix[eq3]

Proof

This is proved using the formula for the joint moment generating function of the linear transformation of a random vector. The joint moment generating function of X is [eq4]Therefore, the joint moment generating function of Y is[eq5]which is the moment generating function of a multivariate normal distribution with mean $A+Bmu $ and covariance matrix $BVB^{intercal }$. Note that $BVB^{intercal }$ needs to be positive definite in order to be the covariance matrix of a proper multivariate normal distribution, but this is implied by the assumption that $B$ is full-rank. Therefore, Y has a multivariate normal distribution with mean $A+Bmu $ and covariance matrix $BVB^{intercal }$, because two random vectors have the same distribution when they have the same joint moment generating function.

The following examples present some important special cases of the above property.

Example 1 - Sum of two independent normal random variables

The sum of two independent normal random variables has a normal distribution.

Proposition Let X_1 be a normal random variable with mean $mu _{1}$ and variance $sigma _{1}^{2}$. Let X_2 be a random variable, independent of X_1, having a normal distribution with mean $mu _{2}$ and variance $sigma _{2}^{2}$. Then, the random variable[eq6]has a normal distribution with mean [eq7]and variance [eq8]

Proof

First of all, we need to use the fact that mutually independent normal random variables are jointly normal: the $2 imes 1$ random vector X defined as[eq9]has a multivariate normal distribution with mean [eq10]and covariance matrix [eq11]We can write[eq12]where[eq13]Therefore, according to the above proposition on linear transformations, Y has a normal distribution with mean[eq14]and variance[eq15]

Example 2 - Sum of more than two mutually independent normal random variables

The sum of more than two independent normal random variables also has a normal distribution.

Proposition Let [eq16] be K mutually independent normal random variables, having means [eq17] and variances [eq18]. Then, the random variable [eq19]has a normal distribution with mean [eq20]and variance [eq21]

Proof

This can be obtained, either generalizing the proof of the proposition in Example 1, or using the proposition in Example 1 recursively (starting from the first two components of X, then adding the third one and so on).

Example 3 - Linear combinations of mutually independent normal random variables

The properties illustrated in the previous two examples can be further generalized to linear combinations of K mutually independent normal random variables.

Example Let [eq22] be K mutually independent normal random variables, having means [eq23] and variances [eq24]. Let [eq25] be K constants. Then, the random variable Y defined as[eq26]has a normal distribution with mean [eq27]and variance [eq28]

Proof

First of all, we need to use the fact that mutually independent normal random variables are jointly normal: the Kx1 random vector X defined as[eq29]has a multivariate normal distribution with mean [eq30]and covariance matrix [eq31]We can write[eq32]where[eq33]Therefore, according to the above proposition on linear transformations, Y has a (multivariate) normal distribution with mean[eq34]and variance[eq35]

Example 4 - Linear transformation of a normal random variable

Here is a special case of the above proposition on normal random vectors, for the case in which X has dimension $1 imes 1$ (i.e., it is a random variable).

Proposition Let X be a normal random variable with mean mu and variance sigma^2. Let a and $b$ be two constants (with $b eq 0$). Then the random variable Y defined by[eq36]has a normal distribution with mean[eq37]and variance[eq38]

Proof

This is just a special case ($K=1$) of the above proposition on linear transformations of multivariate normal vectors.

Example 5 - Linear combinations of mutually independent normal random vectors

The property illustrated in Example 3 can be generalized to linear combinations of mutually independent normal random vectors.

Proposition Let [eq39] be n mutually independent Kx1 normal random vectors, having means [eq40] and covariance matrices [eq41]. Let [eq42] be n real $L imes K$ full-rank matrices. Then, the $L imes 1$ random vector Y defined as[eq43]has a normal distribution with mean [eq44]and covariance matrix [eq45]

Proof

This is a consequence of the fact that mutually independent normal random vectors are jointly normal: the $Kn imes 1$ random vector X defined as[eq46]has a multivariate normal distribution with mean [eq47]and covariance matrix [eq48]Therefore, we can apply the above proposition on linear transformations to the vector X.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let [eq49]be a $2 imes 1$ multivariate normal random vector with mean [eq50]and covariance matrix[eq51]

Find the distribution of the random variable Z defined as[eq52]

Solution

We can write[eq53]where [eq54]Being a linear transformation of a multivariate normal random vector, Z is also multivariate normal. Actually, it is univariate normal, because it is a scalar. Its mean is[eq55]and its variance is[eq56]

Exercise 2

Let X_1, ..., X_n be n mutually independent standard normal random variables. Let [eq57] be a constant.

Find the distribution of the random variable Y defined as[eq58]

Solution

Being a linear combination of mutually independent normal random variables, Y has a normal distribution with mean[eq59]and variance[eq60]

How to cite

Please cite as:

Taboga, Marco (2021). "Linear combinations of normal random variables", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/probability-distributions/normal-distribution-linear-combinations.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.