IID sequence

by Marco Taboga, PhD

The acronym IID stands for "Independent and Identically Distributed".

A sequence of random variables (or random vectors) is IID if and only if the following two conditions are satisfied:

1. the terms of the sequence are mutually independent;

2. they all have the same probability distribution.

Definition

A formal definition follows.

Of course, the definition applies also to the special case in which the terms of the sequence are uni-dimensional, that is, they are random variables.

Usage in asymptotic theory

The requirement that a sequence of random variables be IID is often found in asymptotic theory, for example to derive some versions of the Law of Large Numbers and of the Central Limit Theorem.

However, the requirement that a sequence be IID is very strong and is often replaced by milder conditions such as covariance stationarity or mixing.

Usage in maximum likelihood estimation

The IID assumption is frequently made also in maximum likelihood estimation, where it helps to write down the joint density of the observed sample. In fact, when the observations in a sample are IID:

• thanks to the independence assumption, the joint density of all the observations is equal to the product of the marginal densities of the single observations;

• thanks to the assumption that all the variables are identically distributed, all the marginal densities in the product are the same.

If want to learn how the IID assumption is used in maximum likelihood (ML) estimation, you could read the following lectures:

• Multivariate normal distribution - ML estimation

• Normal linear regression model - ML estimation

More details

More details about IID sequences are presented in the lecture entitled Sequences of random variables.

Keep reading the glossary

Previous entry: Heteroskedasticity

Next entry: Impossible event

How to cite

Please cite as:

Taboga, Marco (2021). "IID sequence", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/glossary/IID-sequence.