The distribution of a continuous random variable can be characterized through its probability density function (pdf). The probability that a continuous random variable takes a value in a given interval is equal to the integral of its probability density function over that interval, which in turn is equal to the area of the region in the xy-plane bounded by the x-axis, the pdf and the vertical lines corresponding to the boundaries of the interval.
For example, in the picture below the blue line is the pdf of a normal random variable and the area of the red region is equal to the probability that the random variable takes a value comprised between -2 and 2.
The following is a formal definition.
Definition The probability density function of a continuous random variable is a function such thatfor any interval .
The set of values for which is called the support of .
Suppose that a random variable has probability density function
To compute the probability that takes a value in the interval , you need to integrate the probability density function over that interval:
It is important to understand a fundamental difference between the probability density function, which characterizes the distribution of a continuous random variable, and the probability mass function, which characterizes the distribution of a discrete random variable (remember: a random variable is discrete if the number of values it can take is countable, while the number of values that a continuous random variable can take is uncountable). The probability mass function of a discrete variable is a function that gives you, for any real number , the probability that will be equal to . On the contrary, if is a continuous variable, its probability density function evaluated at a given point is not the probability that will be equal to . As a matter of fact, this probability is equal to zero for any becausewhere is any primitive (or indefinite integral) of .
If you are puzzled by the latter result, you are advised to read the lecture on zero-probability events.
Although it is not a probability, the value of the pdf at a given point can be given a straightforward interpretation:where is a small increment.
The proof we are going to give is not rigorous. Rather, we are focusing on the intuition. For the sake of simplicity, we assume that the pdf is a continuous function. Strictly speaking, this is not necessary, although most of the pdfs that are encountered in practice are continuous (by definition, a pdf must be integrable; however, while all continuous functions are integrable, not all integrable functions are continuous). If the pdf is continuous and is small, then is well approximated by for any belonging to the interval . It follows that
In the above approximate equality, we consider the probability that will be equal to or to a value belonging to a small interval near . In particular, we consider the interval . The probability is proportional to the length of the small interval we are considering. The constant of proportionality is the probability density function of evaluated at . Thus, the higher the pdf is at a given point , the higher is the probability that will take a value near .
Related concepts are those of:
joint probability density function, which characterizes the distribution of a continuous random vector;
marginal probability density function, which characterizes the distribution of a subset of entries of a random vector;
conditional probability density function, which is a pdf obtained by conditioning on the realization of another random variable.
Probability density functions are discussed in more detail in the lecture entitled Random variables.
Previous entry: Prior probability
Next entry: Probability mass function
Most of the learning materials found on this website are now available in a traditional textbook format.