In the lecture entitled Conditional probability we have stated a number of properties that conditional probabilities should satisfy to be rational in some sense. We have proved that, whenever , these properties are satisfied if and only ifbut we have not been able to derive a formula for probabilities conditional on zero-probability events, that is, we have not been able to find a way to compute when .
Thus, we have concluded that the above elementary formula cannot be taken as a general definition of conditional probability, because it does not cover zero-probability events.
In this lecture we discuss a completely general definition of conditional probability, which covers also the case in which .
The plan of the lecture is as follows:
We define the concept of a partition of events.
We show that, given a partition of events, conditional probability can be regarded as a random variable (probability conditional on a partition).
We show that, when no zero-probability events are involved, probabilities conditional on a partition satisfy a certain property (the fundamental property of conditional probability).
We require that the fundamental property of conditional probability be satisfied also when zero-probability events are involved and we show that this requirement is sufficient to unambiguously pin down probabilities conditional on a partition. This requirement can therefore be used to give a completely general definition of conditional probability.
Define a partition of events of as follows:
Definition Let be a collection of non-empty subsets of . is called a partition of events of if
all subsets are events;
if then either or ;
In other words, is a partition of events of if it is a division of into non-overlapping and non-empty events that cover all of .
A partition of events of is said to be finite if there are a finite number of sets in the partition; it is said to be infinite if there are an infinite number of sets in the partition; it is said to be countable if the sets in the partition are countable; it is said to be arbitrary if the number of sets in the partition is not necessarily finite or countable (i.e., it can be uncountable).
Example Suppose that we toss a die. Six numbers (from to can appear face up, but we do not yet know which one of them will appear. The sample space isLet any subset of be considered an event. Define the two eventsThen, is a partition of events of . In fact,and . Now define the the three eventsand the collection . is not a partition of events of : while condition 1 and 3 in the definition above are satisfied, condition 2 is not because
Suppose we are given a finite partition of events of , such that for every .
Suppose that we are interested in the probability of a specific event and that at a certain time in the future we will receive some information about the realized outcome . In particular, we will be told to which one of the subsets , , ..., the realized outcome belongs. When we receive the information that the realized outcome belongs to the set , we will update our assessment of the probability of the event , computing its conditional probability Before receiving the information, this conditional probability is unknown and can be regarded as a random variable, denoted by and defined as follows:
is the probability of conditional on the partition .
Example Let us continue with the example above, where the sample space isand is a partition of events of withLet us assign equal probability to all the outcomes:Let us now analyze the conditional probability of the event We haveThe conditional probability is a random variable defined as follows:Since , the probability mass function of is
A fundamental property of is that its expected value equals the unconditional probability :
This is proved as follows:
Example In the previous example, had the following probability mass function:and it is easy to verify the above property:
This property can be generalized as follows:
Proposition (Fundamental property) Let be a finite partition of events of such that for every . Let be any event obtained as a union of events . Let be the indicator function of . Let be defined as above. Then,
Suppose we are not able to explicitly define (the probability of an event conditional on the partition ). This can happen, for example, because contains a zero-probability event and, therefore, we cannot use the formula to define for . Although we are not able to explicitly define , we require, by analogy with the cases in which we are instead able to explicitly define it, that satisfies the fundamental property of conditional probability, that is, for all events obtained as unions of events . But, how can we be sure that there exists a random variable satisfying this property? Existence is guaranteed by the following important theorem, that we state without providing a proof.
Proposition Let be an arbitrary partition of events of . Let be an event. Then there exists at least one random variable that satisfies the property:for all events obtainable as unions of events . Furthermore, if and are two random variables and both satisfy the above property, that is,for all , then and are almost surely equal, that is, there exists a zero-probability event such that
According to the above theorem, a random variable satisfying the fundamental property of conditional probability exists and is unique (up to almost sure equality). As a consequence, we can indirectly define the probability of an event conditional on the partition as . This indirect way of defining conditional probability is summarized in the following definition.
Definition (Probability conditional on a partition) Let be a partition of events of . Let be an event. We say that a random variable is a probability of conditional on the partition iffor all events obtainable as unions of events .
As we have seen above, such a random variable is guaranteed to exist and is unique up to almost sure equality.
This apparently abstract definition of conditional probability is extremely useful. One of its most important applications is the derivation of conditional probability density functions for continuous random vectors (see the lecture entitled Conditional probability distributions).
The following section present more details about conditional probabilities.
In rigorous probability theory, when conditional probability is regarded as a random variable, it is defined with respect to sigma-algebras, rather than with respect to partitions. Let be a probability space. Let be a sub--algebra of , i.e. is a -algebra and . Let be an event. We say that a random variable is a conditional probability of with respect to the -algebra if:It can be shown that this definition is completely equivalent to our definition above, provided is the smallest -algebra containing all the events obtainable as unions of events (where is a partition of events of ).
Let denote the probability of a generic event conditional on the partition . We say that the probability space admits a regular probability conditional on the partition if, for any fixed , is a probability measure on the events , that is, is a probability space for any ,.
Most of the learning materials found on this website are now available in a traditional textbook format.