Autocorrelation

Autocorrelation is the coefficient of linear correlation between two terms of a sequence of random variables.

Autocorrelation is also called serial correlation.

Table of contents

Definition
Autocorrelation and weakly stationary sequences
Sample autocorrelation
Autocorrelation function
ACF plots
Examples
1. Example 1 - ACF of an AR(1) autoregressive process
2. Example 2 - Sample autocorrelation

Definition

The following is formal definition.

Definition Let be a sequence of random variables. The autocorrelation coefficient between two terms of the sequence and $X_{n+k}$ is [eq2]

In other words, the autocorrelation coefficient is just the coefficient of linear correlation between two random variables belonging to the same sequence.

Note that the covariance is called autocovariance.

Autocorrelation and weakly stationary sequences

Remember that a sequence of random variables is said to be covariance stationary (or weakly stationary) if and only if:

all the terms of the sequence have the same mean:
the covariance between any two terms of the sequence depends only on on how far apart they are located from each other, and not on where they are located in the sequence:

The second of these two properties implies that all the random variables in the sequence have the same variance:because .

When a sequence is covariance stationary, the autocorrelation coefficient between two terms of the sequence and $X_{n+k}$ depends only on : [eq8]

We denote it by $ho _{k}$ : [eq9] and we call it autocorrelation at lag (the distance between two terms of the sequence is called lag).

Sample autocorrelation

When we observe the first realizations of a sequence , we can compute the sample autocorrelation at lag : [eq11] where is the sample mean [eq12]

If is covariance stationary, then the numerator of is a consistent estimator of and the denominator is a consistent estimator of . As a consequence, is a consistent estimator of the autocorrelation at lag .

Autocorrelation function

The autocorrelation function (ACF) is the function that maps lags to autocorrelations, that is, $ho _{k}$ is considered as a function of (see the examples below).

When the mapping is from lags to sample autocorrelations , then we call it sample ACF.

ACF plots

An ACF plot is a bar chart (or a line chart) that plots the autocorrelation function:

lags are on the x-axis;
the autocorrelations corresponding to the lags are on the y-axis.

Examples

Let's look at some examples of ACF and ACF plots.

Example 1 - ACF of an AR(1) autoregressive process

Suppose that is a covariance stationary sequence such thatwhere is a constant and is an IID sequence of standard normal random variables (zero mean and unit variance).

Such a sequence is called an autoregressive process of order 1, or AR(1) process (the order is the maximum lag of the sequence on the right hand side of the equation).

Note that [eq22] where we have performed recursive substitutions of $X_{n+k-i}$ with .

By using this expression for $X_{n+k}$ , we can easily derive the autocovariance at lag : [eq24] where: in steps and we have used the bilinearity of the covariance operator and in step we have used the facts that 1) the covariance of a random variable with itself is equal to its variance; 2) the covariance between and $arepsilon _{n+i}$ is zero for any because $X_{n}$ depends only on $arepsilon _{n+i}$ for and the sequence is IID.

Thus, the autocorrelation at lag is [eq26]

The following ACF plots show the autocorrelation function for different values of .

Example - ACF plots of four different AR(1) processes

Example 2 - Sample autocorrelation

In this example, we show what a sample ACF looks like.

We generate, via Monte Carlo simulations, 200 realizations for each of the four AR(1) processes whose ACFs have been plotted above. The realizations are plotted below.

Example - plots of four simulated AR(1) processes

We then compute their sample ACFs, which are plotted below.

Example - Sample ACFs of four different AR(1) processes

These are the sample versions of the ACFs shown in Example 1. As the sample autocorrelations are noisy estimates of the true autocorrelations, these ACFs do not coincide with those shown in Example 1.

How to cite

Please cite as:

Taboga, Marco (2021). "Autocorrelation", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-statistics/autocorrelation.