Marginal and conditional distributions of a multivariate normal vector

This lecture discusses how to derive the marginal and conditional distributions of one or more entries of a multivariate normal vector.

Table of contents

The multivariate normal vector
Partition of the vector
Partition of the parameters
Normality of the sub-vectors
Independence of the sub-vectors
Schur complement
Determinant of a block matrix
Factorization of joint density functions
Partition of the precision matrix
Distributions conditional on realizations

The multivariate normal vector

A random vector is multivariate normal if its joint probability density function iswhere:

is a mean vector;
is a covariance matrix.

Partition of the vector

We partition into two sub-vectors $X_{a}$ and $X_{b}$ such that [eq2]

The sub-vectors $X_{a}$ and $X_{b}$ have dimensions $K_{a} imes 1$ and $K_{b} imes 1$ respectively. Moreover, $K_{a}+K_{b}=K$ .

Partition of the parameters

We partition the mean vector and the covariance matrix as follows: [eq3] and [eq4]

where:

is the mean of $X_{a}$ ;
is the mean of $X_{b}$ ;
is the covariance matrix of $X_{a}$ ;
is the covariance matrix of $X_{b}$ ;
is the cross-covariance between $X_{a}$ and $X_{b}$ .

Normality of the sub-vectors

The following proposition states that the marginal distributions of the two sub-vectors are also multivariate normal.

Proposition Both $X_{a}$ and $X_{b}$ have a multivariate normal distribution: [eq10]

Proof

The random vector $X_{a}$ can be written as a linear transformation of :where is a $K_{a} imes K$ matrix whose entries are either zero or one. Thus, $X_{a}$ has a multivariate normal distribution because it is a linear transformation of the multivariate normal random vector and multivariate normality is preserved by linear transformations (see the lecture on Linear combinations of normal random variables). Also $X_{b}$ has a multivariate normal distribution because it can be written as a linear transformation of :where is a $K_{b} imes K$ matrix whose entries are either zero or one.

Independence of the sub-vectors

The following proposition states a necessary and sufficient condition for the independence of the two sub-vectors.

Proposition $X_{a}$ and $X_{b}$ are independent if and only if $V_{ab}=0$ .

Proof

$X_{a}$ and $X_{b}$ are independent if and only if their joint moment generating function is equal to the product of their individual moment generating functions (see the lecture entitled Joint moment generating function). Since $X_{a}$ is multivariate normal, its joint moment generating function isThe joint moment generating function of $X_{b}$ isThe joint moment generating function of $X_{a}$ and $X_{b}$ , which is just the joint moment generating function of , is [eq15] from which it is obvious that if and only if $V_{ab}=0$ .

Schur complement

In order to derive the conditional distributions, we are going to rely on the following results, demonstrated in the lecture on Schur complements.

Proposition Let $V_{a}$ be invertible. Let $V/V_{a}$ be the Schur complement of $V_{a}$ in , defined asIf $V/V_{a}$ is invertible, then is invertible and [eq18]

Proposition Let $V_{b}$ be invertible. Let $V/V_{b}$ be the Schur complement of $V_{b}$ in , defined asIf $V/V_{b}$ is invertible, then is invertible and [eq20]

Determinant of a block matrix

We will also need the following results on the determinant of a block matrix.

Proposition If $V_{a}$ is invertible, then

Proposition If $V_{b}$ is invertible, then

Factorization of joint density functions

Another important result that we are going to use concerns the factorization of joint density functions.

Write the joint density of the multivariate normal vector as [eq23]

Suppose that we are able to find a factorizationsuch that is a valid probability density function every time that we fix $x_{b}$ and we see as a function of $x_{a}$ .

Then, [eq27] where:

is the conditional probability density function of $X_{a}$ given $X_{b}=x_{b}$ ;
is the marginal density of $X_{b}$ .

Similarly, if we find a factorization such that is a valid probability density function every time that we fix $x_{a}$ and we see as a function of $x_{b}$ , then [eq33]

Partition of the precision matrix

The blocks of the inverse of the covariance matrix (known as precision matrix) are denoted as follows: [eq34]

Distributions conditional on realizations

We are now ready to derive the conditional distributions.

Proposition Suppose that $V_{b}$ and its Schur complement in are invertible. Then, conditional on $X_{b}=x_{b}$ , the vector $X_{a}$ has a multivariate normal distribution with meanand covariance matrix

Proof

First, define [eq37] and note that [eq38] where: in step we have used the partition of the precision matrix $H=V^{-1}$ ; in step we have used the formulae for the blocks of the precision matrix based on the Schur complements; in step we have definedandWe can now factorize the joint density of $X_{a}$ and $X_{b}$ : [eq41] where is the density of a multivariate normal vector with mean $mu _{a}^{st }$ and covariance matrix $V_{a}^{st }$ , and is the density of a multivariate normal vector with mean $mu _{b}$ and covariance matrix $V_{b}$ .

Proposition Suppose that $V_{a}$ and its Schur complement in are invertible. Then, conditional on $X_{a}=x_{a}$ , the vector $X_{b}$ has a multivariate normal distribution with meanand covariance matrix

Proof

Analogous to the previous proof.

How to cite

Please cite as:

Taboga, Marco (2021). "Marginal and conditional distributions of a multivariate normal vector", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/probability-distributions/multivariate-normal-distribution-partitioning.