Rank & Column Space

When a transformation squishes 3D space onto a plane, or a plane onto a line -- what's the dimensionality of the output? That's the rank.

You've seen transformations stretch, rotate, and shear the plane. But some transformations do something more drastic: they collapse dimensions. A 2D plane might get flattened onto a single line. A 3D volume might get squished onto a flat surface. Rank counts how many dimensions survive.

This turns out to be one of the most useful things you can know about a matrix. Does it lose information? How much? What's the shape of the output? Rank answers all of these.

Full rank: nothing lost

Start with a transformation that keeps the plane fully two-dimensional. Both basis vectors land in independent directions -- they don't collapse onto the same line. The output of this transformation can reach any point in 2D space.

The matrix $\begin{bmatrix} 2 & 1 \\ 0 & 1.5 \end{bmatrix}$ sends $\hat{\imath}$ to $(2, 0)$ and $\hat{\jmath}$ to $(1, 1.5)$. These two vectors point in genuinely different directions, so the image of the transformation spans the full plane. The parallelogram they form has nonzero area.

Rank 2 -- the output fills the full 2D plane. Both basis vectors are independent after the transformation.

This is a full-rank transformation. Every point in the plane is reachable as an output. If you feed in every possible input vector, you can hit any target in 2D. No dimension was collapsed. No information was destroyed.

Rank 1: collapse to a line

Now consider a different transformation. What if both basis vectors land on the same line?

The matrix $\begin{bmatrix} 2 & 1 \\ 1 & 0.5 \end{bmatrix}$ sends $\hat{\imath}$ to $(2, 1)$ and $\hat{\jmath}$ to $(1, 0.5)$. Notice something: $(1, 0.5)$ is exactly half of $(2, 1)$. They point in the same direction. The entire plane gets squished onto a single line through the origin.

Rank 1 -- the entire plane collapses to a single line. Both columns point in the same direction.

Every input vector, no matter where it starts, gets mapped to some point on that gold line. The two-dimensional plane has been crushed down to a one-dimensional line. An entire dimension of information is gone.

Different inputs can now produce the same output. The vectors $(1, 1)$ and $(2, -1)$ both map to $(3, 1.5)$. That's what losing a dimension means: the transformation is no longer one-to-one. You can't tell which input produced a given output.

Rank 0: everything to the origin

The most extreme case. The zero matrix $\begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}$ sends every vector to the origin. Both basis vectors collapse to $(0, 0)$. The entire plane shrinks to a single point.

Rank 0 -- every vector in the plane maps to the origin. All information is destroyed.

This is total annihilation. No matter what you put in, you get the zero vector out. Both dimensions are gone. The "output space" of this transformation is zero-dimensional: a single point.

The formal bit

Those three diagrams illustrate the same idea at different scales. Let's name the concepts.

Column space is the set of all possible outputs of a matrix transformation. Take a matrix $A$ and consider every vector $A\mathbf{v}$ as $\mathbf{v}$ ranges over all possible inputs. The collection of all those outputs is the column space of $A$.

Why "column" space? Because the outputs are all linear combinations of the matrix's columns. When you compute $A\mathbf{v}$, you're combining the columns of $A$ with the entries of $\mathbf{v}$ as weights:

$$A\mathbf{v} = \begin{bmatrix} \mathbf{c}_1 & \mathbf{c}_2 & \cdots & \mathbf{c}_n \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} = v_1\mathbf{c}_1 + v_2\mathbf{c}_2 + \cdots + v_n\mathbf{c}_n$$

So the column space = the span of the matrix's columns = all possible outputs of the transformation.

Rank is the dimension of the column space. It tells you how many independent directions survive the transformation:

• Rank 2 in a 2D transformation: the output is the full plane. Two independent columns.
• Rank 1 in a 2D transformation: the output is a line through the origin. The columns are parallel (one is a scalar multiple of the other).
• Rank 0: the output is just the origin. Both columns are zero vectors.

Full rank means the rank equals the number of columns (for a square matrix, the rank equals the dimension of the space). Nothing gets collapsed. The transformation is invertible -- you can undo it.

Rank-deficient means the rank is less than the maximum possible. Some dimension has been lost. The transformation squishes space, and that squishing is irreversible.

Here's a clean way to think about it: rank tells you how much of the output space is actually reachable. If you have a $3 \times 3$ matrix with rank 2, the output is a 2D plane inside 3D space. A whole dimension of outputs is simply inaccessible.

Worked example: 3D projection

This isn't just theory. You encounter rank-deficient transformations every time you look at your screen.

In 3D graphics, rendering projects 3D objects onto a 2D screen. That projection is a linear transformation from $\mathbb{R}^3$ to $\mathbb{R}^2$ (ignoring perspective for a moment). The projection matrix has three columns, but the output is two-dimensional:

$$P = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}$$

This matrix takes $(x, y, z)$ and outputs $(x, y)$ -- it simply drops the $z$ coordinate. The column space is 2D (the screen plane). The rank is 2. The third column is all zeros: the $z$-axis direction contributes nothing to the output.

What does that mean concretely?

• The point $(3, 5, 10)$ and the point $(3, 5, 200)$ both project to $(3, 5)$. They land on the same pixel.
• You can't tell how far away something is just from its screen position.
• That's why you need a depth buffer. The projection loses the depth dimension, so the renderer stores it separately.

Rank tells you exactly what's lost. A rank-2 projection of 3D space means one dimension is destroyed. You need to track that lost dimension (depth) by other means, or accept that it's gone.

The same principle applies to dimensionality reduction in machine learning. PCA, for instance, deliberately constructs a rank-$k$ approximation of your data matrix -- keeping the $k$ most important directions and throwing away the rest. Rank tells you how many features you kept.

What's next

If rank tells us about the output -- what's reachable, how many dimensions survive -- what about the inputs that get lost? When a transformation squishes the plane onto a line, there are whole families of input vectors that all collapse to the same output. The inputs that map to zero form a special set called the null space. That's where we're headed next.