StatLect
Index > Fundamentals of statistics

Autocorrelation

Autocorrelation is the coefficient of linear correlation between two terms of a sequence of random variables.

Autocorrelation is also called serial correlation.

Table of Contents

Definition

The following is formal definition.

Definition Let [eq1] be a sequence of random variables. The autocorrelation coefficient between two terms of the sequence X_n and $X_{n+k}$ is[eq2]

In other words, the autocorrelation coefficient is just the coefficient of linear correlation between two random variables belonging to the same sequence.

Note that the covariance [eq3] is called autocovariance.

Autocorrelation and weakly stationary sequences

Remember that a sequence of random variables is said to be covariance stationary (or weakly stationary) if and only if:

The second of these two properties implies that all the random variables in the sequence have the same variance:[eq6]because [eq7].

When a sequence is covariance stationary, the autocorrelation coefficient between two terms of the sequence X_n and $X_{n+k}$ depends only on k:[eq8]

We denote it by $
ho _{k}$:[eq9] and we call it autocorrelation at lag k (the distance k between two terms of the sequence is called lag).

Sample autocorrelation

When we observe the first $N$ realizations of a sequence [eq10], we can compute the sample autocorrelation at lag k:[eq11]where $widehat{mu }$ is the sample mean[eq12]

If [eq13] is covariance stationary, then the numerator of [eq14] is a consistent estimator of [eq15] and the denominator is a consistent estimator of [eq16]. As a consequence, [eq17] is a consistent estimator of the autocorrelation at lag k.

Autocorrelation function

The autocorrelation function (ACF) is the function that maps lags to autocorrelations, that is, $
ho _{k}$ is considered as a function of k (see the examples below).

When the mapping is from lags to sample autocorrelations [eq18], then we call it sample ACF.

ACF plots

An ACF plot is a bar chart (or a line chart) that plots the autocorrelation function:

Examples

Let's look at some examples of ACF and ACF plots.

Example 1 - ACF of an AR(1) autoregressive process

Suppose that [eq13] is a covariance stationary sequence such that[eq20]where $
ho $ is a constant and [eq21] is an IID sequence of standard normal random variables (zero mean and unit variance).

Such a sequence is called an autoregressive process of order 1, or AR(1) process (the order is the maximum lag of the sequence on the right hand side of the equation).

Note that [eq22]where we have performed recursive substitutions of $X_{n+k-i}$ with [eq23].

By using this expression for $X_{n+k}$, we can easily derive the autocovariance at lag k:[eq24]where: in steps $rame{A}$ and $rame{B}$ we have used the bilinearity of the covariance operator and in step $rame{C}$ we have used the facts that 1) the covariance of a random variable with itself is equal to its variance; 2) the covariance between X_n and $arepsilon _{n+i}$ is zero for any $i>0$ because $X_{n} $ depends only on $arepsilon _{n+i}$ for $i<0$ and the sequence [eq25] is IID.

Thus, the autocorrelation at lag k is[eq26]

The following ACF plots show the autocorrelation function for different values of $arphi $.

Example - ACF plots of four different AR(1) processes

Example 2 - Sample autocorrelation

In this example, we show what a sample ACF looks like.

We generate, via Monte Carlo simulations, 200 realizations for each of the four AR(1) processes whose ACFs have been plotted above. The realizations are plotted below.

Example - plots of four simulated AR(1) processes

We then compute their sample ACFs, which are plotted below.

Example - Sample ACFs of four different AR(1) processes

These are the sample versions of the ACFs shown in Example 1. As the sample autocorrelations are noisy estimates of the true autocorrelations, these ACFs do not coincide with those shown in Example 1.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.