A subspace is said to be invariant under a linear operator if its elements are transformed by the linear operator into elements belonging to the subspace itself.
The kernel of an operator, its range and the eigenspace associated to the eigenvalue of a matrix are prominent examples of invariant subspaces.
The search for invariant subspaces is one of the most important themes in linear algebra. The reason is simple: as we will see below, the matrix representation of an operator with respect to a basis is greatly simplified (i.e., it becomes block-triangular or block-diagonal) if some of the vectors of the basis span an invariant subspace.
Table of contents
Remember that, given a vector space , a linear operator is a function that preserves linear combinations, that is,for any couple of vectors and any couple of scalars .
Definition Let be a vector space and a linear operator. Let be a subspace of . We say that is invariant under if and only if for any .
In other words, if is invariant under , then the restriction of to , denoted by , is a linear operator on (i.e., ).
Example Let be the space of vectors. Let be the subspace spanned by the vectorIn other words, all the vectors of have the formwhere is a scalar. Suppose that a linear operator is such thatThen, whenever , we have . Therefore, is an invariant subspace under .
The kernel of a linear operator is the subspace
Since and all the elements of are mapped into by the operator , the kernel is invariant under .
The range of a linear operator is the subspace
Since , any is mapped by into . Therefore, the range is invariant.
Let be the space of vectors. Let be a matrix. We can use the matrix to define a linear operator as follows:
Suppose is an eigenvalue of and is the subspace of containing all the eigenvectors associated to (so-called eigenspace).
By the definition of eigenvector, we havefor any . Since is a subspace, . Therefore, the eigenspace is invariant under .
There is a tight link between invariant subspaces and block-triangular matrices.
In order to understand this link, we need to revise some facts about linear operators.
Let be a finite-dimensional vector space and a basis for .
If can be written as a linear combination of the basis asthen its coordinate vector with respect to is
Remember that any operator has an associated matrix, called matrix of the operator with respect to and denoted by , such that, for any , we havewhere and are respectively the coordinate vectors of and with respect to .
We have previously proved that the matrix of the operator has the following structure:
We are now ready to state the main proposition in this lecture.
Proposition Let be a finite-dimensional vector space and a linear operator. Let be a subspace of and a basis for . Complete so as to form a basis for . The subspace is invariant under if and only if has the block-triangular structurewhere the block is , is , is and denotes a block of zeros.
We first prove the "only if part", starting from the hypothesis that is invariant. Denote by the -th entry of . Since is invariant, then, for , belongs to and, as a consequence, it can be written as a linear combination of (and enter with zero coefficient in the linear combination). Therefore, for , the coordinate vector of isAs a consequence, when is invariant, the matrix of the operator isWe now prove the "if part", starting from the hypothesis that has the assumed block-triangular structure. Any vector has a coordinate vector of the formwhere is and is . Then,Therefore, . Since this is true for any , is an invariant subspace.
We can also writewhere is the matrix of the restriction of to with respect to the basis .
Remember that is said to be the sum of subspaces , in which case we writeif and only if
A sum of subspaces like the one just shown is said to be a direct sum and it is denoted byif and only if are linearly independent whenever and for .
Direct sums of invariant subspaces have the following important property.
Proposition Let be a linear space. Let and be subspaces of such thatLet and be bases for and respectively (as a consequence, is a basis for ). Let be a linear operator. Then, and are both invariant under if and only if has the block-diagonal structurewhere the blocks and are and respectively.
We first prove the "only if" part, starting from the hypothesis that and are both invariant under . By the properties of direct sums, any vector has a unique representationwhere and . Moreover, has a unique representation in terms of the basis (for ). Therefore, any can be written as a linear combination of the vectors of . In other words, is a basis for . The first columns of areSince is invariant under , implies that . Therefore, for , can be written as a linear combination of the vectors of and the first columns of areSimilarly, we can demonstrate that the remaining columns of areThus,which is a block-diagonal matrix with the structure described in the proposition. We now prove the "if" part, starting from the hypothesis that is block diagonal. Since is block-upper triangular, is invariant by the proposition above on block-upper triangular matrices. Moreover, any vector has a coordinate vector of the formwhere is and is . Then,Therefore, . Since this is true for any , is an invariant subspace.
The previous proposition can be extended, by applying it recursively, to the case in which and all the subspaces are invariant.
Proposition Let be a linear space. Let , , ..., be subspaces of , with bases , ..., , and such thatso that, as a consequence, is a basis for . Let be a linear operator. Then, all the sets (for ) are invariant under if and only if has the block-diagonal structure
What are the practical implications of everything that we have shown so far? In particular, what happens when we are dealing with linear operators defined by matrices? We provide some answers in this section.
Let be a matrix. Let be the space of all column vectors.
We consider the linear operator defined by the matrix , that is,for any .
Suppose that we have been able to find two invariant subspaces and such that
In other words,and is linearly independent from whenever , and the two vectors are non-zero.
We can choose bases and for and respectively and we know that is a basis for .
Define the following matrices by adjoining the vectors of the basis:
Note that where are vectors that are guaranteed to exist because for by the invariance of , which means that can be written as a linear combination of the basis of (i.e., ). In order to match the notation used in the propositions above, we defineso that
Similarly, we can find a matrix such that
As a consequence, we haveorwhere is invertible because its columns are vectors of a basis, which are by definition linearly independent.
Recall the definition of matrix similarity. The last equation means that is similar to the block-diagonal matrixand the change-of-basis matrix used in the similarity transformation is .
Thus, the process of similarity transformation of a matrix into a block-diagonal matrix (generalized here to the case of more than two invariant subspaces) works as follows:
we identify invariant subspaces such that
we find bases for the invariant subspaces and we use them to construct the matrixwhere the columns of are the vectors of the basis of (for );
we perform the similarity transformationand the matrix turns out to be block-diagonal. In particular, there are blocks on the diagonal and the dimensions of the blocks are equal to the number of columns of the matrices (i.e., the number of vectors in each of the bases).
This is one of the most important workflows in linear algebra! We encourage the reader to solidly understand and memorize it.
We have explained above that the eigenspace associated to an eigenvalue of is an invariant subset.
Denote by the distinct eigenvalues of and by their respective eigenspaces.
As explained in the lecture on the linear independence of eigenvectors, when is not defective, we can form a matrixwhere the columns of are a basis for and all the columns of together are a basis for the space of all column vectors. As a consequence,
Thus, we can use the matrix of eigenvectors to perform a similarity transformation and obtain the block-diagonal matrix
Actually, in the lecture on matrix diagonalization, we have proved that is a diagonal matrix having the eigenvalues on its main diagonal.
Below you can find some exercises with explained solutions.
Define the matrix
Verify thatis an invariant subspace under the linear transformation defined by .
Any vector takes the formwhere is a scalar. Then,As a consequence, and is an invariant subspace.
Let be the space of column vectors. Define the matrix
By simply inspecting , can you find two subspaces and such thatand and are invariant under the linear transformation defined by ?
Note that is a block-diagonal matrix:whereTherefore, two complementary invariant subspaces are
Please cite as:
Taboga, Marco (2021). "Invariant subspace", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/invariant-subspace.
Most of the learning materials found on this website are now available in a traditional textbook format.