StatlectThe Digital Textbook
Index > Fundamentals of probability

Jensen's inequality

Jensens's inequality concerns the expected value of convex and concave transformations of a random variable.

Table of Contents

Table of contents

  1. Statement

  2. Example

    1. Exercise 1

Statement

The following is a formal statement of the inequality.

Proposition Let X be an integrable random variable. Let [eq1] be a convex function such that[eq2]is also integrable. Then, the following inequality, called Jensen's inequality, holds:[eq3]

Proof

A function $g$ is convex if, for any point $x_{0}$ the graph of $g$ lies entirely above its tangent at the point $x_{0}$:[eq4]where $b$ is the slope of the tangent. Setting $x=X$ and [eq5], the inequality becomes[eq6]Taking the expected value of both sides of the inequality and using the fact that the expected value operator preserves inequalities, we obtain[eq7]

If the function $g$ is strictly convex and X is not almost surely constant, then we have a strict inequality:[eq8]

Proof

A function $g$ is strictly convex if, for any point $x_{0}$ the graph of $g$ lies entirely above its tangent at the point $x_{0}$ (and stricly so for points different from $x_{0}$):[eq9]where $b$ is the slope of the tangent. Setting $x=X$ and [eq5], the inequality becomes[eq11]and, of course, [eq12] when [eq13]. Taking the expected value of both sides of the inequality and using the fact that the expected value operator preserves inequalities, we obtain[eq14]where the first inequality is strict because we have assumed that X is not almost surely constant and therefore the event[eq15]does not have probability 1.

If the function $g$ is concave, then[eq16]

Proof

If $g$ is concave, then $-g$ is convex and by Jensen's inequality:[eq17]Multiplying both sides by $-1,$ and using the linearity of the expected value we obtain the result.

If the function $g$ is strictly concave and X is not almost surely constant, then[eq18]

Proof

Similar to previous proof.

Example

Suppose a strictly positive random variable X has expected value[eq19]and it is not constant with probability one. What can we say about the expected value of [eq20], by using Jensen's inequality?

The natural logarithm is a strictly concave function, because its second derivative[eq21] is strictly negative on its domain of definition.

As a consequence, by Jensen's inequality, we have[eq22]

Therefore, [eq20] has a strictly negative expected value.

Exercise 1

Let X be a strictly positive random variable, such that[eq24]What can you infer, using Jensen's inequality, about the following expected value:[eq25]

Solution

The function[eq26]has first derivative[eq27]and second derivative[eq28]The second derivative is strictly negative on the domain of definition of the function. Therefore, the function is strictly concave. Furthermore, X is not almost surely constant, because it has strictly positive variance. Hence, by Jensen's inequality:[eq29]

The book

Most of the learning materials found on this website are now available in a traditional textbook format.