Generalized Eigenvectors

This section deals with defective square matrices (or corresponding linear transformations). Recall that a matrix A is defective if it is not diagonalizable. In other words, a square matrix is defective if it has at least one eigenvalue for which the geometric multiplicity is strictly less than its algebraic multiplicity.
Let T be a linear operator on a (finite dimensional) vector space V. A nonzero vector x in V is called a generalized eigenvector of T corresponding to defective eigenvalue λ if \( \left( \lambda {\bf I} - T \right)^p {\bf x} = {\bf 0} \) for some positive integer p. Correspondingly, we define the generalized eigenspace of T associated with λ:
\[ {\mathrm K}_{\lambda} \left( T \right) = \left\{ {\bf x} \in V \,:\, \left( \lambda {\bf I} - T \right)^p {\bf x} = {\bf 0} \quad\mbox{for some natural number } p \right\} . \]

Note that the generalized eigenspace Kλ consists of the zero vector and all generalized eigenvecors associated with λ, including the eigenspace \( E_{\lambda} \subseteq {\mathrm K}_{\lambda} . \) Recall that a subspace U is T-invariant for a linear transformation T (or a corresponding matrix) if \( T(U) \subseteq U . \) We learn previously that the eigenspace Eλ (the set of all eigenvectors associated with λ and the zero vector) is T-invariant. A similar property holds for the generalized eigenspace.

Theorem: Let T be a linear transformation on a vector space V, and let λ be an eigenvalue of T. Then the generalized eigenspace Kλ is a T-invariant subspace of V, containing Eλ (the eigenspace of T corresponding to λ). ■

Clearly, \( {\bf 0} \in {\mathrm K}_{\lambda} (T) . \) Suppose that vectors x and y are in Kλ. Then there exist natural numbers p and q such that
\[ \left( \lambda \, I - T \right)^p {\bf x} = \left( \lambda \, I - T \right)^q {\bf y} = {\bf 0} . \]
Therefore,
\begin{align*} \left( \lambda \, I - T \right)^{p+q} \left( {\bf x} + {\bf y} \right) &= \left( \lambda \, I - T \right)^{p+q} {\bf x} + \left( \lambda \, I - T \right)^{p+q} {\bf y} \\ &= \left( \lambda \, I - T \right)^{q} {\bf 0} + \left( \lambda \, I - T \right)^{p} {\bf 0} ={\bf 0} . \end{align*}
Hence, their sum \( {\bf x} + {\bf y} \in {\mathrm K}_{\lambda} (T) . \) The proof that Kλ is closed under scalar multiplication is direct, and so is omitted. ■

Notice that if x is a generalized eigenvector of a linear transformation T corresponding to λ and p is the smallest positive integer for which \( \left( \lambda {\bf I} - T \right)^p {\bf x} = {\bf 0} , \) then \( \left( \lambda {\bf I} - T \right)^{p-1} {\bf x} \) is an eigenvector of T associated to λ, and therefore λ is an eigenvalue of T. Therefore, for every eigenvalue λ, we have a chain of null spaces:

\[ E_{\lambda} = \mbox{ker} \left( \lambda {\bf I} - T \right) \subseteq \mbox{ker} \left( \lambda {\bf I} - T \right)^2 \subseteq \cdots \subseteq \mbox{ker} \left( \lambda {\bf I} - T \right)^{p-1} \subseteq \mbox{ker} \left( \lambda {\bf I} - T \right)^{p} = \mbox{ker} \left( \lambda {\bf I} - T \right)^{p+1} = \cdots . \]

The material presented below has little importance for practical applications and serves mostly as a bridge for better understanding topics included in Linear Algebra textbooks published in twenteen century. It is related to so called a Jordan canonical form in honor of the French mathematician Camille Jordan (1838--1922). Eigenvectors and generalized eigenvectors were previously used, for instance, in computation of large powers of square matrices. Now we know better tools (see next chapter) to calculate these powers and actually almost arbitrary function of a square matrix.

For a given eigenvalue of a linear transformation T, the vector x is a generalized eigenvector of rank r if
\[ \begin{split} \left( \lambda {\bf I} - T \right)^{r} {\bf x} &= {\bf 0} \\ \left( \lambda {\bf I} - T \right)^{r-1} {\bf x} &\ne {\bf 0} . \end{split} \]
Given an eigenvalue λ of an n×n matrix A, we say that \( {\bf v}_1 , {\bf v}_2 , \ldots , {\bf v}_r \) form a chain of generalized eigenvectors of length r if \( {\bf v}_1 \ne {\bf 0} \) is an eigenvectors of A and
\[ \begin{split} \left( \lambda {\bf I} - {\bf A} \right) {\bf v}_r &= {\bf v}_{r-1} \\ \left( \lambda {\bf I} - {\bf A} \right) {\bf v}_{r-1} &= {\bf v}_{r-2} , \\ \vdots \qquad& \qquad\vdots \\ \left( \lambda {\bf I} - {\bf A} \right) {\bf v}_{2} &= {\bf v}_{1} , \\ \left( \lambda {\bf I} - {\bf A} \right) {\bf v}_{1} &= {\bf 0} . \end{split} \]

Example: Consider the defective matrix:
\[ {\bf A} = \begin{bmatrix} 15&-6&2 \\ 35&-14&5 \\ 7&-3&2 \end{bmatrix} . \]
Its characteristic polynomial is \( \chi (\lambda ) = \det \left( \lambda {\bf I} - {\bf A} \right) = \left( \lambda -1 \right)^3 . \) Its eigenspace is spanned on two eigenvectors
\[ E_1 = \mbox{Span} \left\{ {\bf u}_1 = \begin{bmatrix} -1 \\ 0 \\ 7 \end{bmatrix} , \quad {\bf u}_2 = \begin{bmatrix} 3 \\ 7 \\ 0 \end{bmatrix} \right\} . \]
Each of these eigenvectors is mapped into a zero vector by the transformation \( {\bf I} - {\bf A} : \)
\[ \left( {\bf I} - {\bf A} \right) {\bf u}_1 = {\bf 0} \qquad \mbox{and} \qquad \left( {\bf I} - {\bf A} \right) {\bf u}_2 = {\bf 0} . \]
Since the eigenspace is 2-dimensional, one can choose other eigenvectors; for instance, instead of vector u1 the vector \( {\bf u}_1 = \left[ 0, 1, 3 \right]^{\mathrm T} \) could be used as well. Therefore, we cannot use these eigenvectors to build the chain of generalized eigenvectors. Nevertheless, we can find a generalized eigenvector that is mapped by \( {\bf I} - {\bf A} \) into a linear combination of eigenvectors u1 and u2. So we need to solve the system of equations
\[ \left( {\bf I} - {\bf A} \right) {\bf v} = a\,{\bf u}_1 + b\, {\bf u}_2 , \]
for some scalars a and b. The augmented matrix is read as
\[ \left[ \begin{array}{ccc|c} -14&6&-2&-a+3b \\ -35&15&-5&7b \\ -7&3&-1&7a \end{array} \right] \,\sim \, \left[ \begin{array}{ccc|c} -14&6&-2&-a+3b \\ 0&0&0&\frac{5}{2}\,a - \frac{1}{2}\, b \\ 0&0&0&\frac{15}{2}\,a - \frac{3}{2}\,b \end{array} \right] . \]
Therefore, we conclude that b = 5a, and we get the vector
\[ {\bf u}_1 + 5\,{\bf u}_2 = \begin{bmatrix} -1 \\ 0 \\ 7 \end{bmatrix} + 5 \begin{bmatrix} 3 \\ 7 \\ 0 \end{bmatrix} = 7 \begin{bmatrix} 2 \\ 5 \\ 1 \end{bmatrix} . \]
Finally, we find the generalized eigenvector x by solving the vector equation
\[ \left( {\bf I} - {\bf A} \right) {\bf x} = 7\,\left[ 2, 5, 1 \right]^{\mathrm T} \qquad \Longrightarrow \qquad {\bf x} = \left[ 1, 0, 0 \right]^{\mathrm T} . \]
A = {{ 15, -6, 2 }, {35, -14, 5}, {7, -3, 2}}
B = IdentityMatrix[3] - A
LinearSolve[B, {{14}, {35}, {7}}]

Example: Consider the matrix
\[ {\bf A} = \begin{bmatrix} -207&296&-68 \\ -358&513&-120 \\ -892&1280&-303 \end{bmatrix} . \]
Its characteristic polynomial is \( \chi (\lambda ) = \det \left( \lambda {\bf I} - {\bf A} \right) = \left( \lambda -1 \right)^3 . \) The given matrix has only one eigenvector \( {\bf u} = \left[ 8, 7, 6 \right]^{\mathrm T} . \) To find the corresponding generalized eigenvectors, we solve the chain equations
\begin{align*} \left( {\bf I} - {\bf A} \right) {\bf v}_2 &= {\bf u} = \left[ 8, 7, 6 \right]^{\mathrm T} \qquad \Longrightarrow \qquad {\bf v}_2 = \left[ 23, 16, 0 \right]^{\mathrm T} , \\ \left( {\bf I} - {\bf A} \right) {\bf v}_3 &= {\bf v}_2 = \left[ 23, 16, 0 \right]^{\mathrm T} \qquad \Longrightarrow \qquad {\bf v}_3 = \left[ 320, 223, 0 \right]^{\mathrm T} . \end{align*}
These three vectors u and v2, v3 span the three dimensional space \( \mathbb{R}^3 . \) ■

Example: Let us consider the matrix
\[ {\bf A} = \begin{bmatrix} 48&-30&-14&1 \\ 65&-41&-19&0 \\ 17&-10&-5&3 \\ -35&22&10&0 \end{bmatrix} . \]
We learn previously that its chrachetristic polynomial \( \chi (\lambda ) = \det \left( \lambda {\bf I} - {\bf A} \right) = \left( \lambda -1 \right)^3 \left( \lambda +1 \right) \) has two eigenvalues λ = -1 and λ = 1 to which correspond two eigenvectors:
\[ {\bf u} = \begin{bmatrix} 7 \\ 9 \\ 5 \\ -3 \end{bmatrix} \qquad\mbox{and} \qquad {\bf v} = \begin{bmatrix} -1 \\ -2 \\ 1 \\ 1 \end{bmatrix} , \]
respectively. To find two generalized eigenvectors, we need to solve two equations:
\[ \left( {\bf I} - {\bf A} \right) {\bf v}_2 = {\bf v} \qquad\mbox{and} \qquad \left( {\bf I} - {\bf A} \right) {\bf v}_3 = {\bf v}_2 . \]
Mathematica easily solve these equations:
B = IdentityMatrix[4]-A
v = {{-1}, {-2}, {1}, {1}}
LinearSolve[B, v]
LinearSolve[B, {{0}, {-1}, {2}, {0}}]
Therefore, we find two generalized eigenvectors
\[ {\bf v}_2 = \begin{bmatrix} 0 \\ -1 \\ 2 \\ 0 \end{bmatrix} \qquad\mbox{and} \qquad {\bf v}_3 = \begin{bmatrix} -2 \\ -5 \\ 4 \\ 0 \end{bmatrix} . \]
The vectors v, v2, and v3 form the chain of generalized eigenvectors of length 3.

Now suppose that we need to find A100 x, where

\[ {\bf x} = \begin{bmatrix} 2 \\ -2 \\ 22 \\ -2 \end{bmatrix} = 2\begin{bmatrix} 1 \\ -1 \\ 11 \\ -1 \end{bmatrix} = {\bf u} + {\bf v} + 2{\bf v}_2 + 3{\bf v}_3 . \]

Then
\[ {\bf A}^{100} {\bf u} + {\bf A}^{100} {\bf v} + 2{\bf A}^{100} {\bf v}_2 + 3{\bf A}^{100} {\bf v}_3 . \]
We know that
\[ {\bf A}\, {\bf u} = -{\bf u} , \quad {\bf A}\, {\bf v} = {\bf v} , \quad {\bf A} {\bf v}_2 = {\bf v}_2 -{\bf v} , \quad {\bf A} {\bf v}_3 = {\bf v}_3 - {\bf v} . \]
Then
\[ {\bf A}^{100} {\bf v}_2 = {\bf v}_2 - 100\,{\bf v} \qquad\mbox{and} \qquad {\bf A}^{100} {\bf v}_3 = {\bf v}_3 - 100\,{\bf v} . \]
Therefore,
\begin{align*} {\bf A}^{100} {\bf x} &= (-1)^{100} {\bf u} + {\bf v} + 2\,{\bf A}^{100} {\bf v}_2 + 3\, {\bf A}^{100} {\bf v}_3 \\ &= {\bf u} + {\bf v} + 2\, {\bf v}_2 - 200\, {\bf v} + 3\,{\bf v}_3 - 300\,{\bf v} \\ &= {\bf u} + 2\, {\bf v}_2 + 3\,{\bf v}_3 - 499\,{\bf v} . \end{align*}

Example: Consider the differential operator \( L = {\bf I} + 2\,\texttt{D} + \texttt{D}^2 , \) where \( \texttt{D} = {\text d}/{\text d}x \) is the derivative operator, on the set P2 of polynomials of degree up tp 2. The monomials \( \left\{ 1, x, x^2 \right\} \) form a basis in the set P2. It has dimension 3, and the derivative operator has the following matrix representation:
\[ \texttt{D} = \begin{bmatrix} 0&1&0 \\ 0&0&2 \\ 0&0&0 \end{bmatrix} \qquad \Longrightarrow \qquad \texttt{D} \left( a + bx + cx^2 \right) = \begin{bmatrix} 0&1&0 \\ 0&0&2 \\ 0&0&0 \end{bmatrix} \begin{bmatrix} a \\ b \\ c \end{bmatrix} \left\{ \begin{array}{c} 1 \\ x \\ x^2 \end{array} \right\} = \begin{bmatrix} b \\ 2c \\ 0 \end{bmatrix} \left\{ \begin{array}{c} 1 \\ x \\ x^2 \end{array} \right\} . \]
Then the linear differential operator has the matrix form:
\[ {\bf L} = {\bf I } + 2 \texttt{D} + \texttt{D}^2 = \begin{bmatrix} 1&0&0 \\ 0&1&0 \\ 0&0&1 \end{bmatrix} + 2 \begin{bmatrix} 0&1&0 \\ 0&0&2 \\ 0&0&0 \end{bmatrix} + \begin{bmatrix} 0&0&2 \\ 0&0&0 \\ 0&0&0 \end{bmatrix} = \begin{bmatrix} 1&2&2 \\ 0&1&4 \\ 0&0&1 \end{bmatrix} . \]
The characteristic polynomial for matrix L is \( \chi (\lambda ) = (\lambda -1 )^3 , \) with one eigenvalue λ = 1. To this eigenvalue corresponds only one eigenvector \( {\bf u} = \left[ 1, 0, 0 \right]^{\mathrm T} \) and two generalized eigenvectors \( {\bf v}_2 = \left[ 0, 1, 0 \right]^{\mathrm T} \) and \( {\bf v}_3 = \left[ 0, 1, -1 \right]^{\mathrm T} \) that are solutions (up to a multiple constant) of the following chain equations:
\[ \left( {\bf I} - {\bf L} \right) {\bf v}_2 = {\bf u} \qquad \mbox{and} \qquad \left( {\bf I} - {\bf L} \right) {\bf v}_3 = {\bf v}_2 . \qquad ■ \]