Linear Combinations & Span

The Question

You learned in Chapter 1 that you can scale a vector and add two vectors together. Now put both operations together: scale each vector by some amount, then add the results. That's called a linear combination.

Here's the question that should nag at you: if you have two vectors, what's the full set of points in space you can reach by scaling and adding them?

Not just one combination. All combinations. Every possible pair of scalars. What shape does that set trace out?

The answer is called the span of those vectors, and it tells you everything about what your vectors can do.

Building Combinations

Let's start with two vectors: $\vec{v}_1 = \begin{bmatrix} 2 \\ 1 \end{bmatrix}$ in blue and $\vec{v}_2 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$ in green. A linear combination looks like $c_1 \vec{v}_1 + c_2 \vec{v}_2$ for any scalars $c_1$ and $c_2$.

Pick a few specific values of $c_1$ and $c_2$ and plot where you land.

Each orange dot is a linear combination $c_1 \vec{v}_1 + c_2 \vec{v}_2$. The dashed lines show each vector's contribution. Scale each vector by any amount, then add.

Those six dots are just six specific combinations. But you can choose any real numbers for $c_1$ and $c_2$. Infinitely many choices, infinitely many points. Where do they all land?

When Vectors Are Parallel

Before answering that, let's look at a simpler case. What if your two vectors point in the same direction?

Take $\vec{v}_1 = \begin{bmatrix} 2 \\ 1 \end{bmatrix}$ and $\vec{v}_2 = \begin{bmatrix} 4 \\ 2 \end{bmatrix}$. Notice that $\vec{v}_2 = 2\vec{v}_1$ -- they're parallel.

When both vectors point in the same direction, every combination lands on a single line through the origin. Scaling $\vec{v}_2$ just gives you a differently scaled version of $\vec{v}_1$. The span is one-dimensional.

Any combination $c_1 \vec{v}_1 + c_2 \vec{v}_2$ simplifies to $c_1 \vec{v}_1 + c_2 (2\vec{v}_1) = (c_1 + 2c_2)\vec{v}_1$. That's just a single scalar times $\vec{v}_1$. You're stuck on one line no matter what you do. The second vector gives you no new reach.

The Full Span of Two Non-Parallel Vectors

Now back to the interesting case. When $\vec{v}_1$ and $\vec{v}_2$ point in different directions, something powerful happens. You can reach any point in the entire 2D plane.

Think about it: pick any target point. You need to solve $c_1 \vec{v}_1 + c_2 \vec{v}_2 = \text{target}$ for $c_1$ and $c_2$. Because the vectors aren't parallel, they form two independent "knobs" you can dial -- one pushes you along $\vec{v}_1$'s direction, the other along $\vec{v}_2$'s. Between the two, you can steer to any destination.

Two non-parallel vectors span the entire 2D plane. The light orange shading represents every reachable point. You can reach any position by dialing the right pair of scalars.

This is the key insight. Two non-parallel vectors in 2D give you full coverage. One vector gives you a line. Two non-parallel vectors give you the whole plane. Adding more vectors to the mix won't expand your reach any further -- you already have everywhere.

The Redundant Vector

So what happens when you throw in a third vector? If the first two already span the whole plane, the third one must land somewhere in that plane. It's already reachable. It adds nothing.

Let's say $\vec{v}_3 = \begin{bmatrix} 3 \\ 3 \end{bmatrix}$. Can we write $\vec{v}_3$ as a combination of $\vec{v}_1$ and $\vec{v}_2$? Yes: $\vec{v}_3 = 1\vec{v}_1 + 1\vec{v}_2 = \begin{bmatrix} 2 \\ 1 \end{bmatrix} + \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 3 \\ 3 \end{bmatrix}$.

$\vec{v}_3$ is already reachable from $\vec{v}_1$ and $\vec{v}_2$. The dashed lines show the parallelogram: go along $\vec{v}_1$ then $\vec{v}_2$, or $\vec{v}_2$ then $\vec{v}_1$ -- either way you arrive at $\vec{v}_3$. The third vector is redundant -- it doesn't expand the span.

This is called linear dependence. The set $\{\vec{v}_1, \vec{v}_2, \vec{v}_3\}$ is linearly dependent because one of them is a combination of the others. It's a passenger, not a driver.

The Formal Bit

Time to put precise language on these ideas.

Linear Combination

A linear combination of vectors $\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n$ is any expression of the form:

$$c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_n\vec{v}_n$$

where $c_1, c_2, \ldots, c_n$ are real numbers (called scalars or coefficients). You scale each vector, then add them up.

Span

The span of a set of vectors is the collection of all linear combinations you can form:

$$\text{span}(\vec{v}_1, \vec{v}_2) = \{c_1\vec{v}_1 + c_2\vec{v}_2 \mid c_1, c_2 \in \mathbb{R}\}$$

In plain English: span is everywhere you can reach.

• One non-zero vector spans a line through the origin.
• Two non-parallel vectors in 2D span the entire plane.
• Three non-coplanar vectors in 3D span all of 3D space.

Linear Dependence and Independence

A set of vectors is linearly dependent if at least one vector in the set can be written as a linear combination of the others. Equivalently, there exist scalars $c_1, c_2, \ldots, c_n$, not all zero, such that:

$$c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_n\vec{v}_n = \vec{0}$$

A set is linearly independent if no vector is redundant. Each one expands the reach. Removing any one of them shrinks the span.

The visual test: if a vector already lives inside the span of the others, it's dependent. If it points "out of" the current span into a new dimension, it's independent.

Worked Example: RGB Color Mixing

Here's a concrete application every programmer knows. Think of colors as vectors in 3D space, where the three axes are Red, Green, and Blue intensity (each from 0 to 1).

Define three color vectors:

$$\vec{R} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \quad \vec{G} = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \quad \vec{B} = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$$

What does the span of just Red and Green look like? It's every combination $c_1 \vec{R} + c_2 \vec{G}$ -- a flat plane in 3D color space. You get:

• $1\vec{R} + 1\vec{G} = (1, 1, 0)$ -- yellow
• $0.8\vec{R} + 0.3\vec{G} = (0.8, 0.3, 0)$ -- dark orange
• $0\vec{R} + 0.5\vec{G} = (0, 0.5, 0)$ -- dark green

Lots of colors, but no blues, purples, or anything with a blue component. You're stuck on a plane.

Add Blue. Now $\text{span}(\vec{R}, \vec{G}, \vec{B})$ is all of 3D color space. Every displayable color is a combination of these three. Blue is linearly independent of Red and Green -- it points in a direction you couldn't reach before.

Now define Yellow: $\vec{Y} = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} = 1\vec{R} + 1\vec{G}$.

Does adding Yellow as a fourth "basis color" let you mix any new colors? No. Yellow is already in the span of Red and Green. It's linearly dependent. $\text{span}(\vec{R}, \vec{G}, \vec{B}, \vec{Y}) = \text{span}(\vec{R}, \vec{G}, \vec{B})$ -- the same set. Adding a redundant vector changes nothing.

This is exactly why monitors use three primary colors: three linearly independent vectors span 3D color space. A fourth primary would be redundant.

// In code: any color is a linear combination of RGB
const R = [1, 0, 0];
const G = [0, 1, 0];
const B = [0, 0, 1];

// Yellow = R + G — already in span(R, G)
const yellow = R.map((v, i) => 1 * v + 1 * G[i] + 0 * B[i]);
// [1, 1, 0]

// Purple needs Blue — not in span(R, G) alone
const purple = R.map((v, i) => 0.5 * v + 0 * G[i] + 0.5 * B[i]);
// [0.5, 0, 0.5]

What's Next

We know what combinations of vectors look like and how to tell whether a vector set is redundant. But what if we had a machine that takes every vector in space and moves it somewhere new -- systematically? Stretching, rotating, squishing the whole plane at once, while keeping grid lines straight and evenly spaced? That's a linear transformation, and it's the subject of Chapter 3.