Orthogonal Matrices & Gram-Schmidt

Some transformations preserve distances and angles perfectly -- no stretching, no squishing. These are orthogonal matrices, and they're built from orthonormal vectors.

Most matrices distort space. They stretch some directions, compress others, skew angles. But a special class of matrices does none of that. An orthogonal matrix transforms space rigidly -- it can rotate and reflect, but every distance and every angle comes through unchanged. If you apply an orthogonal transformation to a shape, it comes out looking exactly like the original, just moved.

This isn't just elegant mathematics. Orthogonal matrices are everywhere in practice: rotation matrices in 3D graphics, the $Q$ factor in QR decomposition, the basis vectors of a camera's coordinate system, and the orthonormal bases used in signal processing. Understanding them -- and knowing how to build them with Gram-Schmidt -- is essential.

Orthogonal matrices preserve the unit circle

The best way to see what orthogonal matrices do is to watch them act on the unit circle. Take every point at distance 1 from the origin and transform it. An orthogonal matrix keeps every point at distance 1 -- the circle stays a perfect circle. A non-orthogonal matrix distorts it into an ellipse.

On the left, a 45-degree rotation (orthogonal). On the right, a stretch by 2 along $x$ and compression by 0.5 along $y$ (not orthogonal). The rotation preserves the circle perfectly. The scaling destroys it.

Left: a rotation matrix maps the unit circle to itself -- distances are perfectly preserved. Right: a non-orthogonal scaling matrix stretches the circle into an ellipse, distorting all distances.

This is the defining visual property of orthogonal matrices. They transform space rigidly. The unit circle is the set of all vectors with length 1, and an orthogonal matrix maps every length-1 vector to another length-1 vector. No direction gets stretched or compressed.

Q^TQ = I visually

What makes a matrix orthogonal? Its columns must be orthonormal -- each column is a unit vector, and any two distinct columns are perpendicular. When you stack these column vectors into a matrix $Q$, the product $Q^TQ$ gives you the identity matrix.

Why? The $(i,j)$ entry of $Q^TQ$ is the dot product of column $i$ and column $j$. If the columns are orthonormal, each column dotted with itself gives 1 (they're unit vectors), and any column dotted with a different column gives 0 (they're perpendicular). That's exactly the identity matrix.

Here are the two columns of a rotation matrix -- $q_1$ and $q_2$ -- shown as vectors. They have length 1, and the right angle between them is marked explicitly.

The columns of $Q$ are unit vectors at right angles. When you compute $Q^TQ$, each diagonal entry is a column dotted with itself (giving 1), and each off-diagonal entry is a column dotted with a different column (giving 0). That's the identity matrix.

This is why orthogonal matrices are so convenient: $Q^{-1} = Q^T$. You never need to compute an inverse the hard way. Just transpose. This makes orthogonal matrices computationally cheap to invert -- a free operation in terms of data, since transposing just reinterprets the same numbers.

Gram-Schmidt step 1: the projection

Now the practical question: how do you build an orthonormal basis? You start with whatever vectors you have -- possibly skewed, non-perpendicular, different lengths -- and systematically straighten them out.

The Gram-Schmidt process does this one vector at a time. Start with two linearly independent vectors $v_1 = (2, 1)$ and $v_2 = (1, 2)$. They're not perpendicular (their dot product is $2 \cdot 1 + 1 \cdot 2 = 4 \neq 0$) and they're not unit length.

Step 1: Keep $v_1$ as is (we'll call it $u_1$). Then compute the projection of $v_2$ onto $u_1$:

$$\text{proj}_{u_1}(v_2) = \frac{v_2 \cdot u_1}{u_1 \cdot u_1} \, u_1 = \frac{4}{5}(2, 1) = (1.6, 0.8)$$

This projection is the component of $v_2$ that lies along $u_1$ -- the part we need to remove to make $v_2$ perpendicular.

The orange vector is the projection of $v_2$ onto $u_1$ -- the "shadow" of $v_2$ along the $u_1$ direction. The dashed line from the tip of $v_2$ down to the projection tip drops at a right angle to $u_1$.

The projection captures how much of $v_2$ is "in the direction of" $u_1$. The rest -- the perpendicular part -- is what we keep.

Gram-Schmidt step 2: subtract and orthogonalize

Step 2: Subtract the projection from $v_2$ to get the perpendicular component:

$$u_2 = v_2 - \text{proj}_{u_1}(v_2) = (1, 2) - (1.6, 0.8) = (-0.6, 1.2)$$

Let's verify: $u_1 \cdot u_2 = 2(-0.6) + 1(1.2) = -1.2 + 1.2 = 0$. They're perpendicular.

The purple vector $u_2$ is perpendicular to $u_1$ -- verified by the zero dot product. It was created by subtracting the projection (the component of $v_2$ along $u_1$) from $v_2$. The gold right-angle marker confirms orthogonality. To finish, normalize both vectors to unit length.

That's the entire Gram-Schmidt algorithm in two dimensions. You take the first vector as-is, project the second onto it, subtract the projection, and you have two perpendicular vectors. Normalize them to unit length and you have an orthonormal basis.

In higher dimensions, the same idea extends naturally: for each new vector $v_k$, subtract its projections onto all previously computed $u_1, u_2, \ldots, u_{k-1}$. What remains is the component perpendicular to all of them.

The formal bit

An orthogonal matrix $Q$ is a square matrix whose columns are orthonormal vectors. This means:

$$Q^TQ = QQ^T = I$$

which immediately gives us $Q^{-1} = Q^T$. The inverse is just the transpose -- no computation needed.

Key properties of orthogonal matrices:

• Preserves lengths: $\|Q\vec{v}\| = \|\vec{v}\|$ for all $\vec{v}$
• Preserves angles: the angle between $Q\vec{u}$ and $Q\vec{v}$ equals the angle between $\vec{u}$ and $\vec{v}$
• Preserves dot products: $(Q\vec{u}) \cdot (Q\vec{v}) = \vec{u} \cdot \vec{v}$
• Determinant is $\pm 1$: $\det(Q) = +1$ for rotations, $\det(Q) = -1$ for reflections

The proofs are clean. For dot product preservation: $(Q\vec{u})^T(Q\vec{v}) = \vec{u}^TQ^TQ\vec{v} = \vec{u}^TI\vec{v} = \vec{u}^T\vec{v}$. Length preservation follows because $\|Q\vec{v}\|^2 = (Q\vec{v})^T(Q\vec{v}) = \vec{v}^T\vec{v} = \|\vec{v}\|^2$.

The Gram-Schmidt process takes a set of linearly independent vectors $\{v_1, v_2, \ldots, v_n\}$ and produces an orthonormal set $\{e_1, e_2, \ldots, e_n\}$:

$$u_1 = v_1, \quad e_1 = \frac{u_1}{\|u_1\|}$$

$$u_k = v_k - \sum_{j=1}^{k-1} \frac{v_k \cdot u_j}{u_j \cdot u_j} \, u_j, \quad e_k = \frac{u_k}{\|u_k\|}$$

Each step subtracts the projections onto all previously computed directions, leaving only the component perpendicular to all of them.

Worked example: building a camera basis

Here's where orthogonal matrices earn their keep in practice. In 3D graphics, every camera needs an orthonormal basis -- three perpendicular unit vectors that define its local coordinate system: forward (where it looks), right (the horizontal axis of the image), and up (the vertical axis of the image). These three vectors form the columns of a rotation matrix that transforms from world coordinates to camera coordinates.

The problem: you're given a look-at direction and a rough world up vector, and you need to produce a clean orthonormal basis.

Suppose the camera is at position $P$ looking at a target $T$. The look direction is:

$$\text{forward} = \text{normalize}(T - P)$$

Let's say this gives us $\text{forward} = (0.6, -0.8, 0)$ -- the camera is looking down and to the right.

Now we need a right vector perpendicular to forward. We use the world's up direction $\text{world\_up} = (0, 1, 0)$ as our starting point. But world_up isn't perpendicular to forward -- the camera is tilted. This is exactly the situation Gram-Schmidt was built for.

Step 1: Compute the right vector.

We need a direction perpendicular to forward in the horizontal plane. The cross product handles this:

$$\text{right} = \text{normalize}(\text{forward} \times \text{world\_up})$$

$$= \text{normalize}\left(\begin{vmatrix} \hat{\imath} & \hat{\jmath} & \hat{k} \\ 0.6 & -0.8 & 0 \\ 0 & 1 & 0 \end{vmatrix}\right) = \text{normalize}(0, 0, 0.6) = (0, 0, 1)$$

Step 2: Compute the true up vector.

The true up must be perpendicular to both forward and right:

$$\text{up} = \text{right} \times \text{forward} = (0, 0, 1) \times (0.6, -0.8, 0)$$

$$= (0 \cdot 0 - 1 \cdot (-0.8),\; 1 \cdot 0.6 - 0 \cdot 0,\; 0 \cdot (-0.8) - 0 \cdot 0.6) = (0.8, 0.6, 0)$$

This is already unit length. Our orthonormal basis is:

$$\text{right} = (0, 0, 1), \quad \text{up} = (0.8, 0.6, 0), \quad \text{forward} = (0.6, -0.8, 0)$$

You can verify: every pair has dot product zero, and every vector has length 1. The view matrix is the orthogonal matrix with these as rows (or columns, depending on convention):

$$V = \begin{bmatrix} 0 & 0 & 1 \\ 0.8 & 0.6 & 0 \\ 0.6 & -0.8 & 0 \end{bmatrix}$$

Since $V$ is orthogonal, $V^{-1} = V^T$ -- and you get the inverse view matrix for free. This is why game engines and graphics APIs use orthonormal bases everywhere: the math stays clean and the inversions stay cheap.

This same pattern shows up whenever you need a local coordinate frame: constructing tangent-space bases for normal mapping, building coordinate frames for physics simulations, or setting up reference frames for robot joints. Gram-Schmidt (or its cross-product shortcut in 3D) is how you get there.

What's next

We've built the theoretical foundation. Now let's put it all to work -- starting with the practical mechanics of 2D transformations in computer graphics.