Eigenvalues & Eigenvectors

Directions that the transformation merely scales — the skeleton you build everything else on.

Suggested next → The Schrödinger Equation · PHYS · T5

The brief

Most directions in space, when a linear transformation is applied to them, change: a rotation sends them somewhere else, a shear tilts them, a projection drops them onto a subspace. But for almost every linear transformation there are a handful of special directions that survive — directions the transformation merely scales without rotating or reflecting them. These are the eigenvectors, and the scaling factors are the eigenvalues. The German prefix eigen- (own, characteristic, proper) was attached by David Hilbert around 1904. Once you know a transformation's eigenvectors and eigenvalues, you know almost everything about it.

Formally, an eigenvector of a linear transformation T is a nonzero vector v such that T(v) = λv for some scalar λ — the corresponding eigenvalue. Geometrically, T merely stretches v by the factor λ (negative λ flips the direction; complex λ adds rotation in 2D real space). To find them, you solve the characteristic equation det(T − λI) = 0 — a polynomial whose roots are exactly the eigenvalues. Each eigenvalue corresponds to an eigenspace of vectors that all scale by the same factor. The spectral theorem — one of the most consequential results in linear algebra — says that a real symmetric matrix has a complete orthonormal basis of eigenvectors with real eigenvalues. Equivalently, every symmetric matrix can be diagonalized: written as PDP⁻¹ where D is a diagonal matrix of eigenvalues and P's columns are eigenvectors. In eigenvector coordinates, the transformation is diagonal — it acts independently on each axis, which is the simplest possible behavior. Generalizations include Jordan normal form (when full diagonalization fails), singular value decomposition (eigenvalues for non-square matrices, the workhorse of numerical linear algebra), and spectral decomposition in infinite-dimensional Hilbert spaces (the foundation of quantum mechanics). The deep observation is that eigenvalues are intrinsic: they don't depend on the basis you used to write the matrix, only on the transformation itself. They are what the transformation truly is, abstracted from coordinates.

Why nowPrincipal Component Analysis — the oldest and still most-used dimensionality-reduction technique — finds the directions of greatest variance in a dataset by computing the top eigenvectors of the covariance matrix. PageRank finds web pages of greatest importance as the dominant eigenvector of a hyperlink-graph matrix. Quantum mechanics identifies physical observables (position, momentum, energy, spin) with linear operators, and the eigenvalues of those operators are the values that measurements can return. Vibration modes of a structure (whether a building under earthquake load or a guitar string) are eigenvalues of a stiffness matrix. Markov chains converge to a stationary distribution given by the eigenvector with eigenvalue 1. Spectral graph theory studies networks through the eigenvalues of their adjacency or Laplacian matrices.

Further readingStrang and Axler again form the textbook spine. For eigenvalue intuition specifically, Trefethen and Bau is the working numerical analyst's reference. The graph-spectral and PageRank applications are best encountered in Spielman's lecture notes on spectral graph theory (Yale, freely available). For the eigenvalue-as-frequency perspective bridging to PDEs, Strang's Differential Equations and Linear Algebra (2014) braids the two stories together more cleanly than any other text.