Given two discrete random variables (or random vectors) and , their joint probability mass function can be factorized into:
the conditional probability mass function of given ;
the marginal probability mass function of .
The next propositon provides a formal statement of the factorization.
Proposition Let be a discrete random vector with support and joint probability mass function . Denote by the conditional probability mass function of given and by the marginal probability mass function of . Then,for any and .
See the lecture entitled Conditional probability distributions.
If we need to derive the two factors from the joint probability mass function, we usually perform two steps:
marginalize by summing it over all possible values of and obtain the marginal probability mass function ;
divide by and obtain the conditional probability mass function (this can be done only if ).
When the first step (marginalization) is too hard to perform, it is possible to avoid it thanks to a guess-and-verify procedure. The following proposition shows how.
Proposition Suppose there are two functions and such that
for any and , the following holds:
for any fixed , , considered as a function of , is a probability mass function.
Then,
We exploit the fact that the marginal probability mass function of needs to satisfyUsing this property in conjunction with property 1 in the proposition, we obtainThe last equality is a consequence of the fact that, for any fixed , , considered as a function of , is a probability mass function and the sum of a probability mass function over its support equals . Thus,Since we also have thatthen, by necessity, it must be that
Thus, the guess-and verify procedure works as follows. First, we express the joint probability mass function as the product of two factors (this is the "guess" part). Then, we verify that:
one factor (a function of and ) is a probability mass function in for all values of ;
the other factor (a function of ) does not depend on .
Example Let be a random vector having a multinomial distribution with parameters , and (the probabilities of the three possible outcomes of each trial) and (the number of trials). The probabilities are strictly positive numbers such thatThe support of isThe joint probability mass function isNote thatTherefore, the joint probability mass function can be factorized aswhereandBut, for any , is the probability mass function of a multinomial distribution with parameters , and . Therefore,
Most learning materials found on this website are now available in a traditional textbook format.