Search for probability and statistics terms on Statlect
StatLect
Index > Fundamentals of probability

Conditional probability

by , PhD

A conditional probability is a probability assigned to an event after receiving some information about other relevant events.

Table of Contents

Aims

The aims of this lecture are to:

The mathematical setting

Let Omega be a sample space and let [eq1] denote the probability assigned to the events $Esubseteq Omega $.

Suppose that, after assigning probabilities [eq2] to the events in Omega, we receive new information about the things that will happen (the possible outcomes).

In particular, suppose that we are told that the realized outcome will belong to a set $Isubseteq Omega $.

How should we revise the probabilities assigned to the events in Omega, to properly take the new information into account?

The answer to this question is provided by the conditional probability [eq3], the revised probability assigned to an event $Esubseteq Omega $ after learning that the realized outcome will be an element of I.

A gradual approach

Despite being an intuitive concept, conditional probability is quite difficult to define in a mathematically rigorous way.

We take a gradual approach in this lecture:

The case of equally likely sample points

Suppose that a sample space Omega has a finite number n of sample points:[eq4]

Suppose also that each sample point is assigned the same probability:[eq5]

Counting favorable cases

In such a simple space, the probability of a generic event E is obtained as[eq6]where $QTR{rm}{card}$ denotes the cardinality of a set, that is, the number of its elements.

In other words, the probability of an event E is obtained in two steps:

  1. counting the number of "cases that are favorable to the event E", that is, the number of elements $omega _{i}$ belonging to E;

  2. dividing the number thus obtained by the number of "all possible cases", that is, the number of elements $omega _{i}$ belonging to Omega.

For example, if [eq7], then[eq8]

First proof of conditional probability formula

When we learn that the realized outcome will belong to a set $Isubseteq Omega $, we still apply the rule[eq9]

However, the number of all possible cases is now equal to the number of elements of I because only the outcomes belonging to I are still possible.

Furthermore, the number of favorable cases is now equal to the number of elements of $Ecap I$ because the outcomes in $Ecap I^{c}$ are no longer possible.

As a consequence,[eq10]

By dividing numerator and denominator by [eq11], we obtain[eq12]

Therefore, when all sample points are equally likely, conditional probabilities are computed as[eq13]

Example

Let us make an example.

Suppose that we toss a die. Six numbers (from 1 to $6)$ can appear face up, but we do not yet know which one of them will appear.

The sample space is[eq14]

Each of the six numbers is a sample point and is assigned probability $frac{1}{6}$.

Define the event E as follows:[eq15]where the event E could be described as "an odd number appears face up".

Now define the event I as follows:[eq16]where the event I could be described as "a number greater than three appears face up".

The probability of I is[eq17]Suppose we are told that the realized outcome will belong to I. How do we have to revise our assessment of the probability of the event E, according to the rules of conditional probability?

First of all, we need to compute the probability of the event $Ecap I$:[eq18]

Then, the conditional probability of E given I is[eq19]

Comments about the formula

In the next section, we will show that the conditional probability formula[eq13]is valid also for more general cases (i.e., when the sample points are not all equally likely).

However, this formula already allows us to understand why defining conditional probability is a challenging task.

In the conditional probability formula, a division by [eq21] is performed. This division is impossible when I is a zero-probability event (i.e., [eq22]).

If we want to be able to define [eq23] also when [eq24], then we need to give a more complicated definition of conditional probability. We will return to this point later.

A more general approach

In this section we give a more general definition of conditional probability, by taking an axiomatic approach.

First, we list the mathematical properties that we would like conditional probability to satisfy.

Then, we prove that the conditional probability formula introduced above satisfies these properties.

Mathematical properties

The conditional probability [eq25] is required to satisfy the following properties:

  1. Probability measure. [eq23] has to satisfy all the properties of a probability measure.

  2. Sure thing. [eq27].

  3. Impossible events. If $Esubseteq I^{c}$ ($I^{c}$, the complement of I with respect to Omega, is the set of all elements of Omega that do not belong to I), then [eq28].

  4. Constant likelihood ratios on I. If $Esubseteq I$, $Fsubseteq I$ and [eq29], then[eq30]

These properties are very intuitive:

  1. Property 1 requires that also conditional probability measures satisfy the fundamental properties that any other probability measure needs to satisfy.

  2. Property 2 says that the probability of a sure thing must be 1: since we know that only things belonging to the set I can happen, then the probability of I must be 1.

  3. Property 3 says that the probability of an impossible thing must be 0: since we know that things not belonging to the set I will not happen, then the probability of the events that are disjoint from I must be 0.

  4. Property 4 is a bit more complex: it says that if $Fsubseteq I$ is - say - two times more likely than $Esubseteq I$ before receiving the information I, then F remains two times more likely than E, also after receiving the information because all the things in E and F remain possible (can still happen) and, hence, there is no reason to expect that the ratio of their likelihoods changes.

Second proof of conditional probability formula

When the above properties are satisfied, then the conditional probability formula can be used.

Proposition Whenever [eq31], [eq32] satisfies the four above properties if and only if[eq13]

Proof

We first show that[eq34]satisfies the four properties whenever [eq35]. As far as property 1) is concerned, we have to check that the three requirements for a probability measure are satisfied. The first requirement for a probability measure is that [eq36]. Since [eq37], by the monotonicity of probability we have that[eq38]Hence,[eq39]Furthermore, since [eq40] and [eq41], also [eq42]The second requirement for a probability measure is that [eq43]. This is satisfied because[eq44]The third requirement for a probability measure is that for any sequence of disjoint sets [eq45] the following holds:[eq46]But,[eq47]so that also the third requirement is satisfied. Property 2) is trivially satisfied:[eq48]Property 3) is verified because, if $Esubseteq I^{c}$, then[eq49]Property 4) is verified because, if $Esubseteq I$, $Fsubseteq I$ and [eq50], then[eq51]So, the "if" part has been proved. Now we prove the "only if" part. We prove it by contradiction. Suppose there exist another conditional probability [eq52] that satisfies the four properties. Then, there exists an event E, such that[eq53]It can not be that $Esubseteq I$, otherwise we would have[eq54]which would be a contradiction, since if [eq55] was a conditional probability it would satisfy:[eq56]If $E $is not a subset of I then [eq57] implies also [eq58] because[eq59]and[eq60]but this would also lead to a contradiction because [eq61].

Tackling division by zero

In the previous section we have generalized the concept of conditional probability. However, we have not been able to define the conditional probability [eq25] for the case in which [eq24]. This case is discussed in the lectures:

The law of total probability

We end this lecture by stating an important formula that allows us to write the probability of an event as a weighted sum of conditional probabilities.

Let $I_{1}$, ..., $I_{n}$ be n events having the following characteristics:

  1. they are mutually disjoint: [eq64] whenever $j eq k$;

  2. they cover all the sample space:[eq65]

  3. they have strictly positive probability: [eq66] for any $j$.

The events $I_{1}$, ..., $I_{n}$ are called a partition of Omega.

The law of total probability states that, for any event E, the following holds:[eq67]which can, of course, also be written as[eq68]

Proof

The law of total probability is proved as follows:[eq69]

Solved exercises

Some solved exercises on conditional probability can be found below.

Exercise 1

Consider a sample space Omega comprising three possible outcomes $omega _{1}$, $omega _{2}$, $omega _{3}$:[eq70]

Suppose the three possible outcomes are assigned the following probabilities:[eq71]

Define the events[eq72]

and denote by $E^{c}$ the complement of E.

Compute [eq73], the conditional probability of F given $E^{c}$.

Solution

We need to use the conditional probability formula[eq74]

The numerator is[eq75]and the denominator is[eq76]

As a consequence,[eq77]

Exercise 2

Consider a sample space Omega comprising four possible outcomes $omega _{1}$, $omega _{2}$, $omega _{3}$, $omega _{4}$:[eq78]

Suppose the four possible outcomes are assigned the following probabilities:[eq79]

Define two events[eq80]

Compute [eq81], the conditional probability of E given F.

Solution

We need to use the formula[eq82]

But[eq83]while, by using additivity, we obtain[eq84]

Therefore,[eq85]

Exercise 3

The Census Bureau has estimated the following survival probabilities for men:

  1. probability that a man lives at least 70 years: 80%;

  2. probability that a man lives at least 80 years: 50%.

What is the conditional probability that a man lives at least 80 years given that he has just celebrated his 70th birthday?

Solution

Given an hypothetical sample space Omega, define the two events[eq86]

We need to find the following conditional probability:[eq87]

The denominator is known:[eq88]

As far as the numerator is concerned, note that $Fsubseteq E$ (if you live at least 80 years then you also live at least 70 years). But $Fsubseteq E$ implies[eq89]

Therefore,[eq90]

Thus,[eq91]

How to cite

Please cite as:

Taboga, Marco (2021). "Conditional probability", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/conditional-probability.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.