StatlectThe Digital Textbook
Index > Fundamentals of probability

Conditional probability as a random variable

In the lecture entitled Conditional probability we have stated a number of properties that conditional probabilities shoud satisfy to be rational in some sense. We have proved that, whenever [eq1], these properties are satisfied if and only if[eq2]but we have not been able to derive a formula for probabilities conditional on zero-probability events, i.e. we have not been able to find a way to compute [eq3] when [eq4].

Thus, we have concluded that the above elementary formula cannot be taken as a general definition of conditional probability, because it does not cover zero-probability events.

In this lecture we discuss a completely general definition of conditional probability, which covers also the case in which [eq5].

The plan of the lecture is as follows:

  1. We define the concept of a partition of events.

  2. We show that, given a partition of events, conditional probability can be regarded as a random variable (probability conditional on a partition).

  3. We show that, when no zero-probability events are involved, probabilities conditional on a partition satisfy a certain property (the fundamental property of conditional probability).

  4. We require that the fundamental property of conditional probability be satisfied also when zero-probability events are involved and we show that this requirement is sufficient to unambiguously pin down probabilities conditional on a partition. This requirement can therefore be used to give a completely general defintion of conditional probability.

Table of Contents

Partitions of events

Let Omega be a sample space and let [eq6] denote the probability assigned to the events $Esubseteq Omega $.

Define a partition of events of Omega as follows:

Definition Let G be a collection of non-empty subsets of Omega. G is called a partition of events of Omega if

  1. all subsets $Gin QTR{cal}{G}$ are events;

  2. if [eq7] then either $G=F$ or $Gcap F=emptyset $;

  3. [eq8].

In other words, G is a partition of events of Omega if it is a division of Omega into non-overlapping and non-empty events that cover all of Omega.

A partition of events of Omega is said to be finite if there are a finite number of sets $G$ in the partition; it is said to be infinite if there are an infinite number of sets $G$ in the partition; it is said to be countable if the sets $G$ in the partition are countable; it is said to be arbitrary if the number of sets $G$ in the partition is not necessarily finite or countable (i.e., it can be uncountable).

Example Suppose that we toss a die. Six numbers (from 1 to $6)$ can appear face up, but we do not yet know which one of them will appear. The sample space is[eq9]Let any subset of Omega be considered an event. Define the two events[eq10]Then, [eq11] is a partition of events of Omega. In fact,[eq12]and [eq13]. Now define the the three events[eq14]and the collection [eq15]. $QTR{cal}{F}$ is not a partition of events of Omega: while condition 1 and 3 in the definition above are satisfied, condition 2 is not because[eq16]

Probabilities conditional on a partition

Suppose we are given a finite partition [eq17] of events of Omega, such that [eq18] for every i.

Suppose that we are interested in the probability of a specific event $Esubseteq Omega $ and that at a certain time in the future we will receive some information about the realized outcome $overline{omega }$. In particular, we will be told to which one of the n subsets $G_{1}$, $G_{2}$, ..., $G_{n}$ the realized outcome $overline{omega }$ belongs. When we receive the information that the realized outcome belongs to the set $G_{i}$, we will update our assessment of the probability of the event E, computing its conditional probability: [eq19]Before receiving the information, this conditional probability is unknown and can be regarded as a random variable, denoted by [eq20] and defined as follows:[eq21]

[eq20] is the probability of E conditional on the partition G.

Example Let us continue with the example above, where the sample space is[eq23]and [eq11] is a partition of events of Omega with[eq25]Let us assign equal probability to all the outcomes:[eq26]Let us now analyze the conditional probability of the event [eq27]We have[eq28]The conditional probability [eq29] is a random variable defined as follows:[eq30]Since [eq31], the probability mass function of [eq20] is[eq33]

The fundamental property of conditional probability

A fundamental property of [eq29] is that its expected value equals the unconditional probability [eq35]:[eq36]

Proof

This is proved as follows:[eq37]

Example In the previous example, [eq29] had the following probability mass function:[eq33]and it is easy to verify the above property:[eq40]

This property can be generalized as follows:

Proposition (Fundamental property) Let [eq41] be a finite partition of events of Omega such that [eq42] for every i. Let H be any event obtained as a union of events [eq43]. Let $1_{H}$ be the indicator function of H. Let [eq20] be defined as above.Then,[eq45]

Proof

Without loss of generality, we can assume that H is obtained as the union of the first k ($kleq n$) sets of the partition G (we can always rearrange the sets $G_{i}$ by changing their indices):[eq46]First note that[eq47]The property is proved as follows:[eq48]

The fundamental property of conditional probability as a defining property

Suppose we are not able to explicitly define [eq49] (the probability of an event E conditional on the partition G). This can happen, for example, because G contains a zero-probability event $G$ and, therefore, we cannot use the formula [eq2] to define [eq51] for $omega in G$. Although we are not able to explicitly define [eq20], we require, by analogy with the cases in which we are instead able to explicitly define it, that [eq29] satisfies the fundamental property of conditional probability, that is,[eq45] for all events H obtained as unions of events $Gin QTR{cal}{G}$. But, how can we be sure that there exists a random variable [eq55] satisfying this property? Existence is guaranteed by the following important theorem, that we state without providing a proof.

Proposition Let G be an arbitrary partition of events of Omega. Let $Ein Omega $ be an event. Then there exists at least one random variable Y that satisfies the property:[eq56]for all events H obtainable as unions of events $Gin QTR{cal}{G}$. Furthermore, if $Y_{1}$ and $Y_{2}$ are two random variables and both satisfy the above property, that is,[eq57]for all H, then $Y_{1}$ and $Y_{2}$ are almost surely equal, that is, there exists a zero-probability event F such that[eq58]

According to the above theorem, a random variable Y satisfying the fundamental property of conditional probability exists and is unique (up to almost sure equality). As a consequence, we can indirectly define the probability of an event E conditional on the partition G as [eq59] Y. This indirect way of defining conditional probability is summarized in the following definition.

Definition (Probability conditional on a partition) Let G be a partition of events of Omega. Let $Ein Omega $ be an event. We say that a random variable [eq60] is a probability of E conditional on the partition G if[eq45]for all events H obtainable as unions of events $Gin QTR{cal}{G}$.

As we have seen above, such a random variable is guaranteed to exist and is unique up to almost sure equality.

This apparently abstract definition of conditional probability is extremely useful. One of its most important applications is the derivation of conditional probability density functions for absolutely continuous random vectors (see the lecture entitled Conditional probability distributions).

More details

The following section present more details about conditional probabilities.

Conditioning with respect to sigma-algebras

In rigorous probability theory, when conditional probability is regarded as a random variable, it is defined with respect to sigma-algebras, rather than with respect to partitions. Let [eq62] be a probability space. Let $QTR{cal}{I}$ be a sub-$sigma $-algebra of $	ciFourier $, i.e. $QTR{cal}{I}$ is a $sigma $-algebra and [eq63]. Let $Ein 	ciFourier $ be an event. We say that a random variable [eq64] is a conditional probability of E with respect to the $sigma $-algebra $QTR{cal}{I,}$ if:[eq65]It can be shown that this definition is completely equivalent to our definition above, provided $QTR{cal}{I}$ is the smallest $sigma $-algebra containing all the events $Hin 	ciFourier $ obtainable as unions of events $Gin QTR{cal}{G}$ (where G is a partition of events of Omega).

Regular conditional probabilities

Let [eq20] denote the probability of a generic event E conditional on the partition G. We say that the probability space [eq67] admits a regular probability conditional on the partition G if, for any fixed omega, [eq68] is a probability measure on the events E, that is, [eq69] is a probability space for any omega,.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.