We review here the basics of computing eigenvalues and eigenvectors. Eigenvalues and eigenvectors play a prominent role in the study of ordinary differential equations and in many applications in the physical sciences. Expect to see them come up in a variety of contexts!
Definitions
Let $A$ be an $n \times n$ matrix. The number $\lambda$ is an eigenvalue of $A$ if there exists a non-zero vector ${\bf v}$ such that $$ A{\bf v} = \lambda {\bf v}. $$ In this case, vector ${\bf v}$ is called an eigenvector of $A$ corresponding to $\lambda$.
Computing Eigenvalues and Eigenvectors
We can rewrite the condition $A{\bf v} = \lambda {\bf v}$ as $$ (A- \lambda I){\bf v} = {\bf 0}. $$ where $I$ is the $n \times n$ identity matrix. Now, in order for a non-zero vector ${\bf v}$ to satisfy this equation, $A – \lambda I$ must not be invertible.
Otherwise, if $A – \lambda I$ has an inverse, \begin{eqnarray*} (A – \lambda I)^{-1}(A – \lambda I){\bf v} & = & (A – \lambda I)^{-1}{\bf 0} \\ {\bf v} & = & {\bf 0}. \end{eqnarray*} But we are looking for a non-zero vector ${\bf v}$. That is, the determinant of $A – \lambda I$ must equal 0. We call $p(\lambda )= \det (A – \lambda I)$ the characteristic
polynomial of $A$. The eigenvalues of $A$ are simply the roots of the characteristic polynomial of $A$.
Example
Let $A = \left[ \begin{array}{rr} 2 & -4\\ -1 & -1 \end{array}\right] $. Then $$\begin{array}{rcl} p(\lambda) & = & \det \left[\begin{array}{cc} 2-\lambda & -4 \\ -1 & -1-\lambda \end{array}\right]\\ & = & (2-\lambda)(-1-\lambda)-(-4)(-1)\\ & = & \lambda^{2} -\lambda -6\\ & = & (\lambda -3)(\lambda +2). \end{array}$$ Thus, $\lambda_1 =3$ and $\lambda_2 = -2$ are the eigenvalues of $A$.
To find eigenvectors ${\bf v} = \left[ \begin{array}{c} v_1\\ v_2\\ \vdots\\ v_n \end{array}\right] $ corresponding to an eigenvalue $\lambda$, we simply solve the system of linear equations given by $$ (A-\lambda I) {\bf v} = {\bf 0}. $$
Example
The matrix $A = \left[ \begin{array}{rr} 2 & -4\\ -1 & -1 \end{array}\right] $ of the previous example has eigenvalues $\lambda_1 =3$ and $\lambda_2 = -2$. Let’s find the eigenvectors corresponding to $\lambda_1 =3$. Let ${\bf v}= \left[{v_1 \atop v_2}\right]$. Then $(A-3I){\bf v}={\bf 0}$ gives us $$ \left[\begin{array}{cc} 2-3 & -4\\ -1 & -1-3 \end{array}\right]\left[\begin{array}{c} v_1\\ v_2 \end{array}\right] = \left[\begin{array}{c} 0\\ 0 \end{array}\right], $$ from which we obtain the duplicate equations \begin{eqnarray*} -v_1-4v_2 & = & 0 \\ -v_1-4v_2 & = & 0. \end{eqnarray*} If we let $v_2=t$, then $v_1=-4t$. All eigenvectors corresponding to $\lambda_1 =3$ are multiples of $\left[{-4 \atop 1}\right] $ and thus the eigenspace corresponding to $\lambda_1 =3$ is given by the span of $\left[{-4 \atop 1}\right] $. That is, $\left\{\left[{-4 \atop 1}\right]\right\}$ is a basis of the eigenspace corresponding to $\lambda_1 =3$.
Repeating this process with $\lambda_2 = -2$, we find that \begin{eqnarray*} 4v_1 -4V_2 & = & 0 \\ -v_1 + v_2 & = & 0 \end{eqnarray*} If we let $v_2=t$ then $v_1=t$ as well. Thus, an eigenvector corresponding to $\lambda_2 = -2$ is $\left[{1 \atop 1}\right]$ and the eigenspace corresponding to $\lambda_2 = -2$ is given by the span of $\left[{1 \atop 1}\right]$. $\left\{\left[{1 \atop 1}\right]\right\}$ is a basis for the eigenspace corresponding to $\lambda_2 = -2$.
In the following example, we see a two-dimensional eigenspace.
Example
Let $A=\left[\begin{array}{ccc} 5 & 8 & 16\\ 4 & 1 & 8\\ -4 & -4 & -11 \end{array}\right]$. Then $p(\lambda ) = \det\left[\begin{array}{ccc} 5-\lambda & 8 & 16\\ 4 & 1-\lambda & 8\\ -4 & -4 & -11-\lambda \end{array}\right] = (\lambda-1)(\lambda+3)^{2}$ after some algebra! Thus, $\lambda_1 = 1$ and $\lambda_2=-3$ are the eigenvalues of $A$. Eigenvectors ${\bf v} = \left[\begin{array}{c} v_1\\ v_2\\ v_3 \end{array}\right]$ corresponding to $\lambda_1=1$ must satisfy
Letting $v_3=t$, we find from the second equation that $v_1=-2t$, and then $v_2=-t$. All eigenvectors corresponding to $\lambda_1=1$ are multiples of $\left[\begin{array}{c} -2\\ -1\\ 1 \end{array}\right]$, and so the eigenspace corresponding to $\lambda_1=1$ is given by the span of $\left[\begin{array}{c} -2\\ -1\\ 1 \end{array}\right]$. $\left\{\left[\begin{array}{c} -2\\ -1\\ 1 \end{array}\right]\right\}$ is a basis for the eigenspace corresponding to $\lambda_1=1$.
Eigenvectors corresponding to $\lambda_2=-3$ must satisfy
The equations here are just multiples of each other! If we let $v_3 = t$ and $v_2 = s$, then $v_1 = -s -2t$. Eigenvectors corresponding to $\lambda_2=-3$ have the form $$ \left[\begin{array}{c} -1\\ 1\\ 0 \end{array}\right]s+\left[\begin{array}{c} -2\\ 0\\ 1 \end{array}\right]t. $$ Thus, the eigenspace corresponding to $\lambda_2=-3$ is two-dimensional and is spanned by $\left[\begin{array}{c} -1\\ 1\\ 0 \end{array}\right]$ and $\left[\begin{array}{c} -2\\ 0\\ 1 \end{array}\right]$. $\left\{\left[\begin{array}{c} -1\\ 1\\ 0 \end{array}\right],\left[\begin{array}{c} -2\\ 0\\ 1 \end{array}\right]\right\}$ is a basis for the eigenspace corresponding to $\lambda_2=-3$.
Notes
- Eigenvalues and eigenvectors can be complex-valued as well as real-valued.
- The dimension of the eigenspace corresponding to an eigenvalue is less than or equal to the multiplicity of that eigenvalue.
- The techniques used here are practical for $2 \times 2$ and $3 \times 3$ matrices. Eigenvalues and eigenvectors of larger matrices are often found using other techniques, such as iterative methods.
Key Concepts
Let $A$ be an $n \times n$ matrix. The eigenvalues of $A$ are the roots of the characteristic polynomial $$ p(\lambda )= \det (A – \lambda I). $$ For each eigenvalue $\lambda$, we find eigenvectors ${\bf v} =\left[ \begin{array}{c} v_1\\ v_2\\ \vdots\\ v_n \end{array}\right] $ by solving the linear system $$ (A – \lambda I){\bf v} = {\bf 0}. $$ The set of all vectors ${\bf v}$ satisfying $A{\bf v}= \lambda {\bf v}$ is called the eigenspace of $A$ corresponding to $\lambda$.