StatlectThe Digital Textbook
Index > Fundamentals of probability

Factorization of joint probability mass functions

Given two discrete random variables (or random vectors) X and Y, their joint probability mass function can be factorized into:

  1. the conditional probability mass function of X given Y=y;

  2. the marginal probability mass function of Y.

The factorization

The next propositon provides a formal statement of the factorization.

Proposition Let [eq1] be a discrete random vector with support $R_{XY}$ and joint probability mass function [eq2]. Denote by [eq3] the conditional probability mass function of X given Y=y and by [eq4] the marginal probability mass function of Y. Then,[eq5]for any x and $y$.

Proof

See the lecture entitled Conditional probability distributions.

A factorization method

If we need to derive the two factors from the joint probability mass function, we usually perform two steps:

  1. marginalize [eq6] by summing it over all possible values of x and obtain the marginal probability mass function [eq7];

  2. divide [eq8] by [eq9] and obtain the conditional probability mass function [eq10] (this can be done only if [eq11]).

When the first step (marginalization) is too hard to perform, it is possible to avoid it thanks to a guess-and-verify procedure. The following proposition shows how.

Proposition Suppose there are two functions [eq12] and $hleft( y
ight) $ such that

  1. for any x and $y$, the following holds:[eq13]

  2. for any fixed $y$, [eq14], considered as a function of x, is a probability mass function.

Then, [eq15]

Proof

We exploit the fact that the marginal probability mass function of Y needs to satisfy[eq16]Using this property in conjunction with property 1 in the proposition, we obtain[eq17]The last equality is a consequence of the fact that, for any fixed $y$, [eq18], considered as a function of x, is a probability mass function and the sum of a probability mass function over its support equals 1. Thus,[eq19]Since we also have that[eq20]then, by necessity, it must be that[eq21]

Thus, the guess-and verify procedure works as follows. First, we express the joint probability mass function as the product of two factors (this is the "guess" part). Then, we verify that:

  1. one factor (a function of x and $y$) is a probability mass function in x for all values of $y$;

  2. the other factor (a function of $y$) does not depend on x.

Example Let X be a $3	imes 1$ random vector having a multinomial distribution with parameters $p_{1}$, $p_{2}$ and $p_{3}$ (the probabilities of the three possible outcomes of each trial) and n (the number of trials). The probabilities are strictly positive numbers such that[eq22]The support R_X of X is[eq23]The joint probability mass function [eq24] is[eq25]Note that[eq26]Therefore, the joint probability mass function can be factorized as[eq27]where[eq28]and[eq29]But, for any $x_{3}leq n$, [eq30] is the probability mass function of a multinomial distribution with parameters $p_{1}/p_{3}$, $p_{2}/p_{3}$ and $n-x_{3}$. Therefore,[eq31]

The book

Most learning materials found on this website are now available in a traditional textbook format.