The vec operator is an operator that transforms a matrix into a column vector by vertically stacking the columns of the matrix.
In this lecture we define the vec operator and we prove some of its most important properties.
We start with a definition.
Definition
Let
be a
matrix. Denote by
the columns of
.
The vectorization of
,
denoted by
,
is the
column
vector
Here is an example of how the vec operator works.
Example
Define the
matrix
Its
vectorization
is
The first two properties of the vec operator are immediate consequences of its definition.
Proposition
If
is a
column vector,
then
Proposition
If
is a
row vector,
then
where
denotes the transpose of
.
The vec operator is linear, that is, it preserves linear combinations.
Proposition
Let
and
be two
matrices and
and
two scalars.
Then
Denote by
the columns of
and by
the columns of
.
By the rules of matrix addition
and
multiplication
of a matrix by a scalar, the
-th
column of
is
Therefore,
Several properties of the vec operator are also properties of the Kronecker product.
Remember that the Kronecker
product
is the block
matrix
where
denotes the
-th
entry of
.
A property of the Kronecker product that we have already proved and that we
will use below is the so-called
mixed-product
property: if
,
,
and
are such that the products
and
are well-defined,
then
The next property concerns outer products, that is, products between a column and a row vector.
Proposition
Let
be a
column vector and
a
row vector.
Then,
Denote the entries of
by
.
Then,
The next property concerns matrix products.
Proposition
Let
be a
matrix and
an
matrix. Denote by
the columns of
.
Then,
Write
as a block
matrix:
By
the rules on the
multiplication of block matrices, we
have
where
each of the products
,
...,
is a column of
.
The stated result then follows by the definition of vectorization.
By using the previous proposition, we can prove the next one.
Proposition
Let
be a
matrix and
an
matrix.
Then,
where
is the
identity matrix.
We
havewhere
in step
we have used the result proved in the previous proposition.
The next property concerns the product
of three matrices. We can think of it as a trick that allows us to free a
matrix
squeezed between two matrices
and
and bring it out of the product.
Proposition
Let
be a
matrix,
an
matrix and
an
matrix.
Then,
Denote the columns of
by
.
The matrix
can be expressed
as
where
is the
-th
vector of the canonical basis of
the
-dimensional
vectors (i.e., an
vector such that its
-th
entry is equal to
and all its other entries are equal to
).
Then, we
have
where:
in steps
and
we have used the linearity of the vec operator; in steps
and
we have used the above result about the vec of outer products; in step
we have used the mixed-product property of the Kronecker product.
Note that the previously derived result
is
a special case of the last result, obtained by setting
.
We can similarly obtain other expressions for
:
Remember that the trace of a matrix is the sum of its diagonal entries.
The proposition below shows a connection between the vec operator and the trace.
Proposition
Let
be a
matrix and
an
matrix.
Then,
where
denotes the trace of
.
This is proved as
follows:
Below you can find some exercises with explained solutions.
Let
be a
matrix,
an
matrix and
an
vector. Prove
that
Since
is a column vector it equal its vec and the vec of its transpose:
Let
be a block matrix
with
blocks
,
and
.
Can you express
in terms of the vec of the blocks?
When we vectorize
we stack its columns vertically starting from the first column on the left and
ending with the last column on the right. Therefore, we first stack all the
columns of
,
then all the columns of
and finally those of
.
Therefore,
Please cite as:
Taboga, Marco (2021). "Vec operator", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/vec-operator.
Most of the learning materials found on this website are now available in a traditional textbook format.