# Eigenvalues and Eigenvectors

Page
Eigenvalues and Eigenvectors – HMC Calculus Tutorial

We review here the basics of computing eigenvalues and eigenvectors. Eigenvalues and eigenvectors play a prominent role in the study of ordinary differential equations and in many applications in the physical sciences. Expect to see them come up in a variety of contexts!

#### Definitions

Let $A$ be an $n \times n$ matrix. The number $\lambda$ is an eigenvalue of $A$ if there exists a non-zero vector ${\bf v}$ such that $$A{\bf v} = \lambda {\bf v}.$$ In this case, vector ${\bf v}$ is called an eigenvector of $A$ corresponding to $\lambda$.

#### Computing Eigenvalues and Eigenvectors

We can rewrite the condition $A{\bf v} = \lambda {\bf v}$ as $$(A- \lambda I){\bf v} = {\bf 0}.$$ where $I$ is the $n \times n$ identity matrix. Now, in order for a non-zero vector ${\bf v}$ to satisfy this equation, $A – \lambda I$ must not be invertible.

Otherwise, if $A – \lambda I$ has an inverse, \begin{eqnarray*} (A – \lambda I)^{-1}(A – \lambda I){\bf v} & = & (A – \lambda I)^{-1}{\bf 0} \\ {\bf v} & = & {\bf 0}. \end{eqnarray*} But we are looking for a non-zero vector ${\bf v}$. That is, the determinant of $A – \lambda I$ must equal 0. We call $p(\lambda )= \det (A – \lambda I)$ the characteristic
polynomial
of $A$. The eigenvalues of $A$ are simply the roots of the characteristic polynomial of $A$.

###### Example

Let $A = \left[ \begin{array}{rr} 2 & -4\\ -1 & -1 \end{array}\right]$. Then $$\begin{array}{rcl} p(\lambda) & = & \det \left[\begin{array}{cc} 2-\lambda & -4 \\ -1 & -1-\lambda \end{array}\right]\\ & = & (2-\lambda)(-1-\lambda)-(-4)(-1)\\ & = & \lambda^{2} -\lambda -6\\ & = & (\lambda -3)(\lambda +2). \end{array}$$ Thus, $\lambda_1 =3$ and $\lambda_2 = -2$ are the eigenvalues of $A$.

To find eigenvectors ${\bf v} = \left[ \begin{array}{c} v_1\\ v_2\\ \vdots\\ v_n \end{array}\right]$ corresponding to an eigenvalue $\lambda$, we simply solve the system of linear equations given by $$(A-\lambda I) {\bf v} = {\bf 0}.$$

###### Example

The matrix $A = \left[ \begin{array}{rr} 2 & -4\\ -1 & -1 \end{array}\right]$ of the previous example has eigenvalues $\lambda_1 =3$ and $\lambda_2 = -2$. Let’s find the eigenvectors corresponding to $\lambda_1 =3$. Let ${\bf v}= \left[{v_1 \atop v_2}\right]$. Then $(A-3I){\bf v}={\bf 0}$ gives us $$\left[\begin{array}{cc} 2-3 & -4\\ -1 & -1-3 \end{array}\right]\left[\begin{array}{c} v_1\\ v_2 \end{array}\right] = \left[\begin{array}{c} 0\\ 0 \end{array}\right],$$ from which we obtain the duplicate equations \begin{eqnarray*} -v_1-4v_2 & = & 0 \\ -v_1-4v_2 & = & 0. \end{eqnarray*} If we let $v_2=t$, then $v_1=-4t$. All eigenvectors corresponding to $\lambda_1 =3$ are multiples of $\left[{-4 \atop 1}\right]$ and thus the eigenspace corresponding to $\lambda_1 =3$ is given by the span of $\left[{-4 \atop 1}\right]$. That is, $\left\{\left[{-4 \atop 1}\right]\right\}$ is a basis of the eigenspace corresponding to $\lambda_1 =3$.

Repeating this process with $\lambda_2 = -2$, we find that \begin{eqnarray*} 4v_1 -4V_2 & = & 0 \\ -v_1 + v_2 & = & 0 \end{eqnarray*} If we let $v_2=t$ then $v_1=t$ as well. Thus, an eigenvector corresponding to $\lambda_2 = -2$ is $\left[{1 \atop 1}\right]$ and the eigenspace corresponding to $\lambda_2 = -2$ is given by the span of $\left[{1 \atop 1}\right]$. $\left\{\left[{1 \atop 1}\right]\right\}$ is a basis for the eigenspace corresponding to $\lambda_2 = -2$.

In the following example, we see a two-dimensional eigenspace.

###### Example

Let $A=\left[\begin{array}{ccc} 5 & 8 & 16\\ 4 & 1 & 8\\ -4 & -4 & -11 \end{array}\right]$. Then $p(\lambda ) = \det\left[\begin{array}{ccc} 5-\lambda & 8 & 16\\ 4 & 1-\lambda & 8\\ -4 & -4 & -11-\lambda \end{array}\right] = (\lambda-1)(\lambda+3)^{2}$ after some algebra! Thus, $\lambda_1 = 1$ and $\lambda_2=-3$ are the eigenvalues of $A$. Eigenvectors ${\bf v} = \left[\begin{array}{c} v_1\\ v_2\\ v_3 \end{array}\right]$ corresponding to $\lambda_1=1$ must satisfy

$\begin{array}{rcrcrcr} 4v_1 & + & 8v_2 & + & 16v_3 & = & 0\\ 4v_1 & & & + & 8v_3 & = & 0\\ -4v_1 & – & 4v_2 & – & 12v_3 & = & 0. \end{array}$

Letting $v_3=t$, we find from the second equation that $v_1=-2t$, and then $v_2=-t$. All eigenvectors corresponding to $\lambda_1=1$ are multiples of $\left[\begin{array}{c} -2\\ -1\\ 1 \end{array}\right]$, and so the eigenspace corresponding to $\lambda_1=1$ is given by the span of $\left[\begin{array}{c} -2\\ -1\\ 1 \end{array}\right]$. $\left\{\left[\begin{array}{c} -2\\ -1\\ 1 \end{array}\right]\right\}$ is a basis for the eigenspace corresponding to $\lambda_1=1$.

Eigenvectors corresponding to $\lambda_2=-3$ must satisfy

$\begin{array}{rcrcrcr} 8v_1 & + & 8v_2 & + & 16v_3 & = & 0\\ 4v_1 & + & 4v_2 & + & 8v_3 & = & 0\\ -4v_1 & – & 4v_2 & – & 8v_3 & = & 0. \end{array}$

The equations here are just multiples of each other! If we let $v_3 = t$ and $v_2 = s$, then $v_1 = -s -2t$. Eigenvectors corresponding to $\lambda_2=-3$ have the form $$\left[\begin{array}{c} -1\\ 1\\ 0 \end{array}\right]s+\left[\begin{array}{c} -2\\ 0\\ 1 \end{array}\right]t.$$ Thus, the eigenspace corresponding to $\lambda_2=-3$ is two-dimensional and is spanned by $\left[\begin{array}{c} -1\\ 1\\ 0 \end{array}\right]$ and $\left[\begin{array}{c} -2\\ 0\\ 1 \end{array}\right]$. $\left\{\left[\begin{array}{c} -1\\ 1\\ 0 \end{array}\right],\left[\begin{array}{c} -2\\ 0\\ 1 \end{array}\right]\right\}$ is a basis for the eigenspace corresponding to $\lambda_2=-3$.

#### Notes

• Eigenvalues and eigenvectors can be complex-valued as well as real-valued.
• The dimension of the eigenspace corresponding to an eigenvalue is less than or equal to the multiplicity of that eigenvalue.
• The techniques used here are practical for $2 \times 2$ and $3 \times 3$ matrices. Eigenvalues and eigenvectors of larger matrices are often found using other techniques, such as iterative methods.

#### Key Concepts

Let $A$ be an $n \times n$ matrix. The eigenvalues of $A$ are the roots of the characteristic polynomial $$p(\lambda )= \det (A – \lambda I).$$ For each eigenvalue $\lambda$, we find eigenvectors ${\bf v} =\left[ \begin{array}{c} v_1\\ v_2\\ \vdots\\ v_n \end{array}\right]$ by solving the linear system $$(A – \lambda I){\bf v} = {\bf 0}.$$ The set of all vectors ${\bf v}$ satisfying $A{\bf v}= \lambda {\bf v}$ is called the eigenspace of $A$ corresponding to $\lambda$.