Search for probability and statistics terms on Statlect
StatLect
Index > Matrix algebra

Vec operator

by , PhD

The vec operator is an operator that transforms a matrix into a column vector by vertically stacking the columns of the matrix.

In this lecture we define the vec operator and we prove some of its most important properties.

Table of Contents

Definition

We start with a definition.

Definition Let A be a $K imes L$ matrix. Denote by [eq1] the columns of A. The vectorization of A, denoted by [eq2], is the $KL imes 1$ column vector[eq3]

Here is an example of how the vec operator works.

Example Define the $2 imes 3$ matrix[eq4]Its vectorization is[eq5]

Vec of column and row vectors

The first two properties of the vec operator are immediate consequences of its definition.

Proposition If $c$ is a Kx1 column vector, then[eq6]

Proposition If $r$ is a $1 imes K$ row vector, then[eq7]where $r^{ op }$ denotes the transpose of $r$.

Linearity

The vec operator is linear, that is, it preserves linear combinations.

Proposition Let A and $B$ be two $K imes L$ matrices and $lpha $ and $eta $ two scalars. Then[eq8]

Proof

Denote by [eq9] the columns of A and by [eq10] the columns of $B$. By the rules of matrix addition and multiplication of a matrix by a scalar, the $l$-th column of $lpha A+eta B$ is [eq11]Therefore,[eq12]

Vec and Kronecker product

Several properties of the vec operator are also properties of the Kronecker product.

Remember that the Kronecker product $Aotimes B$ is the block matrix[eq13]where $A_{kl}$ denotes the $left( k,l ight) $-th entry of A.

A property of the Kronecker product that we have already proved and that we will use below is the so-called mixed-product property: if A, $B$, $C$ and $D$ are such that the products $AC$ and $BD$ are well-defined, then[eq14]

Vec of outer products

The next property concerns outer products, that is, products between a column and a row vector.

Proposition Let $c$ be a Kx1 column vector and $r$ a $1 imes L$ row vector. Then,[eq15]

Proof

Denote the entries of $r$ by [eq16]. Then,[eq17]

Vec of matrix products

The next property concerns matrix products.

Proposition Let A be a $K imes L$ matrix and $B$ an $L imes M$ matrix. Denote by [eq18] the columns of $B$. Then,[eq19]

Proof

Write $B$ as a block matrix:[eq20]By the rules on the multiplication of block matrices, we have[eq21]where each of the products $AB_{ullet 1}$, ..., $AB_{ullet M}$ is a column of $AB$. The stated result then follows by the definition of vectorization.

By using the previous proposition, we can prove the next one.

Proposition Let A be a $K imes L$ matrix and $B$ an $L imes M$ matrix. Then,[eq22]where $I_{M}$ is the $M imes M$ identity matrix.

Proof

We have[eq23]where in step $ rame{A}$ we have used the result proved in the previous proposition.

The next property concerns the product $ABC$ of three matrices. We can think of it as a trick that allows us to free a matrix $B$ squeezed between two matrices A and $C$ and bring it out of the product.

Proposition Let A be a $K imes L$ matrix, $B$ an $L imes M$ matrix and $C$ an $M imes N$ matrix. Then,[eq24]

Proof

Denote the columns of $B$ by [eq25]. The matrix $B$ can be expressed as[eq26]where $e_{m}$ is the $m$-th vector of the canonical basis of the $M$-dimensional vectors (i.e., an $M imes 1$ vector such that its $m$-th entry is equal to 1 and all its other entries are equal to 0). Then, we have[eq27]where: in steps $ rame{A}$ and $ rame{E}$ we have used the linearity of the vec operator; in steps $ rame{B}$ and $ rame{D}$ we have used the above result about the vec of outer products; in step $ rame{C}$ we have used the mixed-product property of the Kronecker product.

Note that the previously derived result [eq28]is a special case of the last result, obtained by setting $C=I_{M}$.

We can similarly obtain other expressions for [eq29]:[eq30]

Vec and trace

Remember that the trace of a matrix is the sum of its diagonal entries.

The proposition below shows a connection between the vec operator and the trace.

Proposition Let A be a $K imes L$ matrix and $B$ an $L imes K$ matrix. Then,[eq31]where [eq32] denotes the trace of $AB$.

Proof

This is proved as follows:[eq33]

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let A be a $K imes L$ matrix, $B$ an $L imes M$ matrix and $r$ an $M imes 1$ vector. Prove that[eq34]

Solution

Since $ABr$ is a column vector it equal its vec and the vec of its transpose: [eq35]

Exercise 2

Let A be a block matrix [eq36]with blocks $B$, $C$ and $D$. Can you express [eq37] in terms of the vec of the blocks?

Solution

When we vectorize A we stack its columns vertically starting from the first column on the left and ending with the last column on the right. Therefore, we first stack all the columns of $B$, then all the columns of $C$ and finally those of $D$. Therefore,[eq38]

How to cite

Please cite as:

Taboga, Marco (2021). "Vec operator", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/vec-operator.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.