The generalized eigenvectors of a matrix are vectors that are used to form a basis together with the eigenvectors of when the latter are not sufficient to form a basis (because the matrix is defective).
Table of contents
We start with a formal definition.
Definition Let be a matrix. Let be an eigenvalue of . A non-zero vector is said to be a generalized eigenvector of associated to the eigenvalue if and only if there exists an integer such thatwhere is the identity matrix.
Note that ordinary eigenvectors satisfy
Therefore, an ordinary eigenvector is also a generalized eigenvector. However, the converse is not necessarily true.
Example Define the matrixIts characteristic polynomial is where in step we have used the Laplace expansion. Thus, the only eigenvalue (with algebraic multiplicity equal to ) isThe vector satisfiesHence, is an eigenvector of . We haveThe vectorsatisfieshence it is a generalized eigenvector.
The following criterion can be used as an equivalent definition of generalized eigenvector.
Proposition Let be a matrix. Let be an eigenvalue of . A non-zero vector is a generalized eigenvector of associated to the eigenvalue if and only if
The "if" part is trivial, as any non-zero vector satisfyingis by definition a generalized eigenvector of . Let us now prove the "only if" part. Let be the space of all vectors. Suppose that a generalized eigenvector satisfiesfor a given integer . As demonstrated in the lecture on matrix powers, the null spacebecomes larger by increasing , but it cannot be larger thanIn other words,for any integer . As a consequence,that is, satisfies
We now define the rank of a generalized eigenvector.
Definition Let be a matrix. Let be an eigenvalue of . Let be a generalized eigenvector of associated to the eigenvalue . We say that is a generalized eigenvector of rank if and only if
Thus, a generalized eigenvector of rank is an ordinary eigenvector.
The set of all generalized eigenvectors associated to an eigenvalue is called a generalized eigenspace.
Definition Let be a matrix. Let be an eigenvalue of . The set of all generalized eigenvectors (plus the zero vector)is called the generalized eigenspace associated to .
Note that we have already proved (see Equivalent definition above) that the null space comprises all the generalized eigenvectors. However, it comprises also the zero vector, which is not a generalized eigenvector.
Since a generalized eigenspace is the null space of a power of , it has two important properties:
it is a linear subspace (as all null spaces are);
it is invariant with respect to the linear transformation defined by (see Null space of a matrix polynomial), that is,whenever .
A consequence of the second point is thatfor any and any .
Two generalized eigenspaces corresponding to two distinct eigenvalues have only the zero vector in common.
Proposition Let be a matrix. Let and be two distinct eigenvalues of (i.e., ). Then, their generalized eigenspaces satisfy
We are going to use the following notation:The proof is by contradiction. Suppose that is a non-zero vector belonging to the intersection of the two generalized eigenspaces. Let be the smallest integer such thatso thatwhich implies that is an eigenvector associated to . Clearly, . Note that is obtained by repeatedly applying to the transformationwhich maps into itself because both and map into itself (the latter by the invariant property discussed above). Thus, not only , but also , that is,Since is an eigenvector, it is non-zero andMoreover, it is an eigenvector associated to , which implies that it cannot be also an eigenvector associated to (because eigenvectors corresponding to distinct eigenvalues are linearly independent). As a consequence,For any , we have orHence, is not a generalized eigenvector associated to . But we have proved above that . Thus, we have arrived at a contradiction. As a consequence, the zero vector is the only vector belonging to the intersection of the two generalized eigenspaces.
Thanks to the things that we have discovered in this lecture, we can improve our understanding of minimal polynomials.
Remember that the minimal polynomial of is the annihilating polynomial (i.e., ) having the lowest possible degree and it can be written in terms of aswhere are the distinct eigenvalues of .
Proposition Let be a matrix. Let be the minimal polynomial of :Then, for , the exponent is the rank of the generalized eigenvectors associated to having the highest rank.
We are going to prove the proposition for the case . The other cases are similar. First of all, we are going to prove that there exists a generalized eigenvector of rank . Defineand note thatbecause otherwise would not be minimal. Hence, there exists a non-zero vector such thatDefineThen,andThus,which implies that is a generalized eigenvector of rank . We now need to prove that there do not exist generalized eigenvectors of higher rank. The proof is by contradiction. Suppose that there exists a generalized eigenvector of higher rank, that is, such thatfor . DefineThen,Factor the minimal polynomial asSince is an annihilating polynomial, we haveDefineThus, which implies that . Butwhich implies that . This is impossible since is non-zero, and and have only the zero vector in common (they can be used to form a direct sum, as demonstrated in the lecture on the Primary Decomposition Theorem; therefore, they must have only the zero vector in common). Hence, we have arrived at a contradiction and the initial assumption that there exists a a generalized eigenvector of rank must be wrong.
Thus, the exponent in the minimal polynomial provides two key pieces of information:
there exists at least one generalized eigenvector of rank associated to ;
no generalized eigenvector associated to can have rank greater than .
A rather important consequence of these two points is thatwhich is proved in detail in a solved exercise at the end of this lecture.
In other words, the generalized eigenspace associated to is the null space of .
We already knew that
But the exponent tells us exactly when null spaces stop growing:where denotes strict inclusion.
Thus, using the terminology introduced in the lectures on the Range null-space decomposition, is the index of the matrix .
Let be the space of all vectors and a matrix.
In a previous lecture we have proved the Primary Decomposition Theorem, which states that the vector space can be written aswhere denotes a direct sum, are the distinct eigenvalues of and are the same strictly positive integers that appear in the minimal polynomial.
As a consequence, by the definition of direct sum, we are able to uniquely write each vector aswhere for .
Thus we can re-interpret / re-state the Primary Decomposition Theorem by using the terminology introduced in this lecture: the vector space can be written as a direct sum of generalized eigenspaces and every vector can be written as a sum of generalized eigenvectors corresponding to distinct eigenvalues.
An immediate consequence of the Primary Decomposition Theorem, as restated above, follows.
Proposition Let be the space of all vectors. Let be a matrix. Then, there exists a basis for formed by generalized eigenvectors of .
Choose a basis for each generalized eigenspace and write each vector in equation (1) as a linear combination of the basis of . Thus, we can write any as a linear combination of generalized eigenvectors, and the union of the bases of the generalized eigenspaces spans . The vectors of the union are linearly independent because is a direct sum of the eigenspaces. Hence, the union is a basis for .
It is interesting to contrast this result with the result discussed in the lecture on the linear independence of eigenvectors: while it is not always possible to form a basis of (ordinary) eigenvectors for , it is always possible to form a basis of generalized eigenvectors!
The dimension of each generalized eigenspace is equal to the algebraic multiplicity of the corresponding eigenvalue.
Proposition Let be a matrix. Let be an eigenvalue of having algebraic multiplicity equal to . Let be the generalized eigenspace associated to . Then, the dimension of is .
By the Schur decomposition theorem, there exists a unitary matrix such thatwhere is upper triangular and denotes the conjugate transpose of . Since and are similar, they have the same eigenvalues. Moreover, the Schur decomposition can be performed in such a way that the last entries on the diagonal of are equal to . We haveandWrite the matrix as a block-triangular matrixwhere is an upper triangular matrix with non-zero entries on its main diagonal, is an upper triangular matrix with zero entries on its main diagonal and denotes a generic matrix of possibly non-zero entries. We haveThe block is upper triangular and its diagonal entries are non-zero, while (by a simple induction argument that is very similar to that used in the proof of the Cayley-Hamilton theorem). The first rows of are clearly linearly independent, while the last are zero. Therefore, the rank of is . Since is full-rank, and multiplication by a full-rank square matrix preserves rank, also has rank . Then, the rank-nullity theorem allows us to obtain the desired result:
Below you can find some exercises with explained solutions.
In an example above we have found two generalized eigenvectors of the matrixCan you find a third generalized eigenvector so as to complete the basis of generalized eigenvectors?
We have already found the generalized eigenvector satisfyingand the generalized eigenvectorsatisfyingNow, we computeand, for example, the vectorsatisfiesMoreover, is a basis for the space of vectors (it is the so-called standard basis).
Let be a matrix. Let be an eigenvalue of and its corresponding exponent in the minimal polynomial. We have proved that there exists at least one generalized eigenvector of rank associated to and no generalized eigenvector associated to can have rank greater than . Explain in detail while these facts imply that
Since is less than or equal to the algebraic multiplicity of and the latter is less than or equal to , we have . Hence, by the properties of matrix powers, Now suppose that , so thatSince no generalized eigenvector can have rank greater than , it must be thatHence, andThe stated result is obtained by combining (2) and (3).
Please cite as:
Taboga, Marco (2021). "Generalized eigenvector", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/generalized-eigenvector.
Most of the learning materials found on this website are now available in a traditional textbook format.