Search for probability and statistics terms on Statlect
StatLect
Index > Fundamentals of probability

Factorization of joint probability mass functions

by , PhD

Given two discrete random variables (or random vectors) X and Y, their joint probability mass function can be factorized into:

  1. the conditional probability mass function of X given Y=y;

  2. the marginal probability mass function of Y.

Table of Contents

The factorization

The next proposition provides a formal statement of the factorization.

Proposition Let [eq1] be a discrete random vector with support $R_{XY}$ and joint probability mass function [eq2]. Denote by [eq3] the conditional probability mass function of X given Y=y and by [eq4] the marginal probability mass function of Y. Then,[eq5]for any x and $y$.

Proof

See the lecture entitled Conditional probability distributions.

A factorization method

If we need to derive the two factors from the joint probability mass function, we usually perform two steps:

  1. marginalize [eq6] by summing it over all possible values of x and obtain the marginal probability mass function [eq7];

  2. divide [eq8] by [eq9] and obtain the conditional probability mass function [eq10] (this can be done only if [eq11]).

When the first step (marginalization) is too hard to perform, it is possible to avoid it thanks to a guess-and-verify procedure. The following proposition shows how.

Proposition Suppose there are two functions [eq12] and $hleft( y ight) $ such that

  1. for any x and $y$, the following holds:[eq13]

  2. for any fixed $y$, [eq14], considered as a function of x, is a probability mass function.

Then, [eq15]

Proof

We exploit the fact that the marginal probability mass function of Y needs to satisfy[eq16]Using this property in conjunction with property 1 in the proposition, we obtain[eq17]The last equality is a consequence of the fact that, for any fixed $y$, [eq18], considered as a function of x, is a probability mass function and the sum of a probability mass function over its support equals 1. Thus,[eq19]Since we also have that[eq20]then, by necessity, it must be that[eq21]

Thus, the guess-and verify procedure works as follows. First, we express the joint probability mass function as the product of two factors (this is the "guess" part). Then, we verify that:

  1. one factor (a function of x and $y$) is a probability mass function in x for all values of $y$;

  2. the other factor (a function of $y$) does not depend on x.

Example Let X be a $3 imes 1$ random vector having a multinomial distribution with parameters $p_{1}$, $p_{2}$ and $p_{3}$ (the probabilities of the three possible outcomes of each trial) and n (the number of trials). The probabilities are strictly positive numbers such that[eq22]The support R_X of X is[eq23]The joint probability mass function [eq24] is[eq25]When [eq26], we have that[eq27]where $inom{n}{x_{3}}$ is a binomial coefficient. Therefore, the joint probability mass function can be factorized as[eq28]where[eq29]and[eq30]But, for any $x_{3}leq n$, [eq31] is the probability mass function of a multinomial distribution with parameters [eq32], [eq33] and $n-x_{3}$. Therefore,[eq34]Note that [eq35] is the pmf of a binomial distribution with parameters n and $p_{3}$.

How to cite

Please cite as:

Taboga, Marco (2021). "Factorization of joint probability mass functions", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/factorization-of-joint-probability-mass-functions.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.