Eigenvectors & Eigenvalues

Most vectors change direction when you apply a transformation. But some special vectors just get scaled -- they stay on their own line. Those are eigenvectors.

We've spent many chapters looking at what transformations do to space. They stretch it, rotate it, shear it, collapse it. We've measured area change with determinants, tracked what survives with rank, and identified what gets destroyed with null spaces. But we haven't asked one of the most revealing questions: are there any vectors that a transformation doesn't knock off course?

It turns out that for most transformations, there are. A few special directions go in, get scaled up or down (or flipped), and come back out pointing the same way. These are eigenvectors, and the scaling factors are eigenvalues. They reveal the skeleton of a transformation -- the directions along which the transformation acts most simply.

Most vectors change direction

Consider the matrix:

$$A = \begin{bmatrix} 2 & 1 \\ 0 & 1 \end{bmatrix}$$

This is a shear-stretch: it doubles the $x$-component and adds the $y$-component to it, while leaving $y$ alone. Let's apply it to several vectors and see what happens.

The faint arrows are the original vectors; the bolder arrows show where they land after applying $A$. Each one points in a different direction than it started. The transformation has knocked them off course.

The vector $(0, 1)$ pointed straight up but landed at $(1, 1)$ -- tilted to the right. The vector $(1, 1)$ was at 45 degrees but landed at $(3, 1)$ -- almost horizontal. The vector $(1, 2)$ was steep but landed at $(4, 2)$ -- much flatter. In every case, the direction shifted. That's what most transformations do to most vectors.

Eigenvectors stay on their line

Now look at the same transformation, but focus on two special directions. The vector $(1, 0)$ -- pointing along the $x$-axis -- and the vector $(-1, 1)$.

Apply $A = \begin{bmatrix} 2 & 1 \\ 0 & 1 \end{bmatrix}$ to each:

$$A \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 2 \\ 0 \end{bmatrix} = 2 \begin{bmatrix} 1 \\ 0 \end{bmatrix}$$

$$A \begin{bmatrix} -1 \\ 1 \end{bmatrix} = \begin{bmatrix} -1 \\ 1 \end{bmatrix} = 1 \begin{bmatrix} -1 \\ 1 \end{bmatrix}$$

The vector $(1, 0)$ got doubled in length but stayed on the $x$-axis. The vector $(-1, 1)$ didn't change at all -- it came back as itself. Both stayed on their own lines. These are the eigenvectors of $A$, with eigenvalues $\lambda = 2$ and $\lambda = 1$.

The purple vectors are eigenvectors. The vector $(1, 0)$ gets stretched to $(2, 0)$ -- doubled in length, same direction. The vector $(-1, 1)$ maps to itself -- unchanged. The gray vector $(1, 2)$ changes direction, like everything else. The faint purple lines show the eigenspaces: directions that the transformation preserves.

This is the core idea. While the transformation twists and shears most of the plane, there are preferred directions where its behavior is simple: just scaling. Finding those directions is what eigenvalue analysis is all about.

Eigenvalue = 2: pure scaling

Let's isolate what an eigenvalue means. Take an eigenvector $\vec{v}$ with eigenvalue $\lambda = 2$. When you apply the transformation, you get $A\vec{v} = 2\vec{v}$. The output is exactly twice the input -- same direction, double the length.

The faint arrow is the original eigenvector $(1, 1)$. The bold arrow is the result after applying the transformation: $(2, 2)$. It's exactly twice as long, pointing in exactly the same direction. The eigenvalue $\lambda = 2$ is just this scaling factor.

The eigenvalue is the answer to a simple question: "by how much does this vector get scaled?" If $\lambda = 2$, the vector doubles. If $\lambda = 3$, it triples. If $\lambda = 0.5$, it shrinks to half. The direction never changes -- that's what makes it an eigenvector.

Eigenvalue = negative: flip and scale

But eigenvalues don't have to be positive. What if $\lambda = -0.5$? The vector gets scaled by $-0.5$: it shrinks to half its length and flips to point the opposite direction. It's still an eigenvector -- it stays on the same line through the origin. It just ends up on the other side.

The faint arrow is the original eigenvector $(2, 1)$. The bold arrow is the result: $(-1, -0.5)$. It flipped direction and shrank to half its length. The eigenvalue $\lambda = -0.5$ captures both effects: the negative sign means "flip", the magnitude $0.5$ means "shrink by half". The vector still lies on the same purple line through the origin.

Negative eigenvalues show up in reflections and certain rotational-like transformations. The key insight is the same: the vector stays on its line. Whether it gets stretched, shrunk, flipped, or any combination -- as long as it doesn't rotate off that line, it's an eigenvector.

The formal bit

An eigenvector of a matrix $A$ is a nonzero vector $\vec{v}$ such that:

$$A\vec{v} = \lambda\vec{v}$$

The scalar $\lambda$ is the corresponding eigenvalue. The equation says: applying $A$ to $\vec{v}$ has the same effect as multiplying $\vec{v}$ by a scalar. The transformation, which could do anything to space, acts on this particular direction as pure scaling.

To find eigenvalues, rearrange the equation:

$$A\vec{v} - \lambda\vec{v} = \vec{0}$$

$$(A - \lambda I)\vec{v} = \vec{0}$$

This says $\vec{v}$ is in the null space of $(A - \lambda I)$. For a nonzero $\vec{v}$ to exist, the matrix $(A - \lambda I)$ must be singular -- its determinant must be zero:

$$\det(A - \lambda I) = 0$$

This is the characteristic equation. For a $2 \times 2$ matrix $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$:

$$\det\begin{bmatrix} a - \lambda & b \\ c & d - \lambda \end{bmatrix} = (a - \lambda)(d - \lambda) - bc = 0$$

This expands to a quadratic in $\lambda$:

$$\lambda^2 - (a + d)\lambda + (ad - bc) = 0$$

The quantity $(a + d)$ is the trace (sum of the diagonal), and $(ad - bc)$ is the determinant. So the characteristic equation is:

$$\lambda^2 - \text{tr}(A) \cdot \lambda + \det(A) = 0$$

Solve the quadratic, and you get the eigenvalues. Plug each eigenvalue back into $(A - \lambda I)\vec{v} = \vec{0}$ to find the eigenvectors.

Let's verify with our matrix $A = \begin{bmatrix} 2 & 1 \\ 0 & 1 \end{bmatrix}$. The trace is $2 + 1 = 3$, and the determinant is $(2)(1) - (1)(0) = 2$:

$$\lambda^2 - 3\lambda + 2 = 0 \quad \Rightarrow \quad (\lambda - 2)(\lambda - 1) = 0$$

So $\lambda = 2$ and $\lambda = 1$ -- exactly the eigenvalues we found geometrically.

Worked example: repeated transformations in animation

In animation and physics simulations, you often apply a transformation repeatedly. Each frame, you multiply by the same matrix. What happens after 10 frames? 100? The answer depends entirely on the eigenvectors and eigenvalues.

Consider a transformation that stretches along one direction and shrinks along another:

$$A = \begin{bmatrix} 2 & 0 \\ 0 & 0.5 \end{bmatrix}$$

This is a diagonal matrix, so the eigenvectors are obvious: $(1, 0)$ with $\lambda = 2$, and $(0, 1)$ with $\lambda = 0.5$.

Now start with an arbitrary vector, say $(1, 1)$. Apply $A$ repeatedly:

After many applications, the vector points almost entirely in the $(1, 0)$ direction -- the eigenvector with the largest eigenvalue. The component along the $\lambda = 0.5$ eigenvector shrinks exponentially ($0.5^n \to 0$), while the component along $\lambda = 2$ grows exponentially ($2^n \to \infty$).

This is why eigenvectors matter in practice. They're the stable directions of a repeated transformation. In a physics simulation, objects stretch along the dominant eigenvector over time. In a Markov chain, the system converges toward the eigenvector with eigenvalue 1. In Google's PageRank, the steady-state distribution of web traffic is the dominant eigenvector of the link matrix.

In code, computing this iteratively looks like:

function applyRepeated(matrix, vector, steps) {
  let v = [...vector];
  for (let i = 0; i < steps; i++) {
    v = [
      matrix[0][0] * v[0] + matrix[0][1] * v[1],
      matrix[1][0] * v[0] + matrix[1][1] * v[1]
    ];
  }
  return v;
}

const A = [[2, 0], [0, 0.5]];
applyRepeated(A, [1, 1], 10); // [1024, ~0.001]

After enough iterations, no matter what vector you started with (as long as it has some component in the dominant eigenvector direction), the result aligns with the eigenvector whose eigenvalue has the largest absolute value. The other directions fade away. That's the power of eigenvalue analysis -- it tells you the long-term behavior of a system without running the simulation.

What's next

Eigenvectors point in directions that stay clean under a transformation. But what makes a set of vectors "clean" in general? When eigenvectors are perpendicular to each other, the math gets especially elegant -- decompositions become simple, and transformations factor into independent scalings along independent axes. That's orthogonality -- perpendicularity and its consequences.