Spectral theorem

In mathematics, particularly linear algebra and functional analysis, a spectral theorem is a result about when a linear operator or matrix can be diagonalized (that is, represented as a diagonal matrix in some basis). This is extremely useful because computations involving a diagonalizable matrix can often be reduced to much simpler computations involving the corresponding diagonal matrix. The concept of diagonalization is relatively straightforward for operators on finite-dimensional vector spaces but requires some modification for operators on infinite-dimensional spaces. In general, the spectral theorem identifies a class of linear operators that can be modeled by multiplication operators, which are as simple as one can hope to find. In more abstract language, the spectral theorem is a statement about commutative C*-algebras. See also spectral theory for a historical perspective.
Examples of operators to which the spectral theorem applies are self-adjoint operators or more generally normal operators on Hilbert spaces.
The spectral theorem also provides a canonical decomposition, called the spectral decomposition, eigenvalue decomposition, or eigendecomposition, of the underlying vector space on which the operator acts.
Augustin-Louis Cauchy proved the spectral theorem for self-adjoint matrices, i.e., that every real, symmetric matrix is diagonalizable. In addition, Cauchy was the first to be systematic about determinants. The spectral theorem as generalized by John von Neumann is today perhaps the most important result of operator theory.
This article mainly focuses on the simplest kind of spectral theorem, that for a self-adjoint operator on a Hilbert space. However, as noted above, the spectral theorem also holds for normal operators on a Hilbert space.
Finite-dimensional case.
Hermitian maps and Hermitian matrices.
We begin by considering a Hermitian matrix on formula_1 (but the following discussion will be adaptable to the more restrictive case of symmetric matrices on formula_2). We consider a Hermitian map on a finite-dimensional complex inner product space endowed with a positive definite sesquilinear inner product formula_3. The Hermitian condition on formula_4 means that for all ,
An equivalent condition is that , where is the Hermitian conjugate of . In the case that is identified with a Hermitian matrix, the matrix of can be identified with its conjugate transpose. (If is a real matrix, this is equivalent to , that is, is a symmetric matrix.)
This condition implies that all eigenvalues of a Hermitian map are real: it is enough to apply it to the case when is an eigenvector. (Recall that an eigenvector of a linear map is a (non-zero) vector such that for some scalar . The value is the corresponding eigenvalue. Moreover, the eigenvalues are roots of the characteristic polynomial.)
Theorem. If is Hermitian, there exists an orthonormal basis of consisting of eigenvectors of . Each eigenvalue is real.
We provide a sketch of a proof for the case where the underlying field of scalars is the complex numbers.
By the fundamental theorem of algebra, applied to the characteristic polynomial of , there is at least one eigenvalue and eigenvector . Then since 
we find that is real. Now consider the space , the orthogonal complement of . By Hermiticity, is an invariant subspace of . Applying the same argument to shows that has an eigenvector . Finite induction then finishes the proof.
The spectral theorem holds also for symmetric maps on finite-dimensional real inner product spaces, but the existence of an eigenvector does not follow immediately from the fundamental theorem of algebra. To prove this, consider as a Hermitian matrix and use the fact that all eigenvalues of a Hermitian matrix are real.
The matrix representation of in a basis of eigenvectors is diagonal, and by the construction the proof gives a basis of mutually orthogonal eigenvectors; by choosing them to be unit vectors one obtains an orthonormal basis of eigenvectors. can be written as a linear combination of pairwise orthogonal projections, called its spectral decomposition. Let
be the eigenspace corresponding to an eigenvalue . Note that the definition does not depend on any choice of specific eigenvectors. is the orthogonal direct sum of the spaces where the index ranges over eigenvalues. 
In other words, if denotes the orthogonal projection onto , and are the eigenvalues of , then the spectral decomposition may be written as
If the spectral decomposition of "A" is formula_9, then formula_10 and formula_11 for any scalar formula_12 It follows that for any polynomial one has
The spectral decomposition is a special case of both the Schur decomposition and the singular value decomposition.
Normal matrices.
The spectral theorem extends to a more general class of matrices. Let be an operator on a finite-dimensional inner product space. is said to be normal if . One can show that is normal if and only if it is unitarily diagonalizable. Proof: By the Schur decomposition, we can write any matrix as , where is unitary and is upper-triangular.
If is normal, one sees that . Therefore, must be diagonal since a normal upper triangular matrix is diagonal (see normal matrix). The converse is obvious.
In other words, is normal if and only if there exists a unitary matrix such that
where is a diagonal matrix. Then, the entries of the diagonal of are the eigenvalues of . The column vectors of are the eigenvectors of and they are orthonormal. Unlike the Hermitian case, the entries of need not be real.
Compact self-adjoint operators.
In the more general setting of Hilbert spaces, which may have an infinite dimension, the statement of the spectral theorem for compact self-adjoint operators is virtually the same as in the finite-dimensional case.
Theorem. Suppose is a compact self-adjoint operator on a (real or complex) Hilbert space . Then there is an orthonormal basis of consisting of eigenvectors of . Each eigenvalue is real.
As for Hermitian matrices, the key point is to prove the existence of at least one nonzero eigenvector. One cannot rely on determinants to show existence of eigenvalues, but one can use a maximization argument analogous to the variational characterization of eigenvalues. 
If the compactness assumption is removed, it is "not" true that every self-adjoint operator has eigenvectors.
Bounded self-adjoint operators.
Possible absence of eigenvectors.
The next generalization we consider is that of bounded self-adjoint operators on a Hilbert space. Such operators may have no eigenvalues: for instance let be the operator of multiplication by on , that is,
Now, a physicist would say that formula_4 "does" have eigenvectors, namely the formula_17, where formula_18 is a Dirac delta-function. A delta-function, however, is not a normalizable function; that is, it is not actually in the Hilbert space . Thus, the delta-functions are "generalized eigenvectors" but not eigenvectors in the strict sense.
Spectral subspaces and projection-valued measures.
In the absence of (true) eigenvectors, one can look for subspaces consisting of "almost eigenvectors". In the above example, for example, where formula_19 we might consider the subspace of functions supported on a small interval formula_20 inside formula_21. This space is invariant under formula_4 and for any formula_23 in this subspace, formula_24 is very close to formula_25. In this approach to the spectral theorem, if formula_4 is a bounded self-adjoint operator, one looks for large families of such "spectral subspaces". Each subspace, in turn, is encoded by the associated projection operator, and the collection of all the subspaces is then represented by a projection-valued measure. 
One formulation of the spectral theorem expresses the operator as an integral of the coordinate function over the operator's spectrum formula_27 with respect to a projection-valued measure.
When the self-adjoint operator in question is compact, this version of the spectral theorem reduces to something similar to the finite-dimensional spectral theorem above, except that the operator is expressed as a finite or countably infinite linear combination of projections, that is, the measure consists only of atoms.
Multiplication operator version.
An alternative formulation of the spectral theorem says that every bounded self-adjoint operator is unitarily equivalent to a multiplication operator. The significance of this result is that multiplication operators are in many ways easy to understand.
Theorem. Let be a bounded self-adjoint operator on a Hilbert space . Then there is a measure space and a real-valued essentially bounded measurable function on and a unitary operator such that
The spectral theorem is the beginning of the vast research area of functional analysis called operator theory; see also the spectral measure.
There is also an analogous spectral theorem for bounded normal operators on Hilbert spaces. The only difference in the conclusion is that now may be complex-valued.
Direct integrals.
There is also a formulation of the spectral theorem in terms of direct integrals. It is similar to the multiplication-operator formulation, but more canonical.
Let formula_4 be a bounded self-adjoint operator and let formula_33 be the spectrum of formula_4. The direct-integral formulation of the spectral theorem associates two quantities to formula_4. First, a measure formula_36 on formula_33, and second, a family of Hilbert spaces formula_38 We then form the direct integral Hilbert space
The elements of this space are functions (or "sections") formula_40 such that formula_41 for all formula_42. 
The direct-integral version of the spectral theorem may be expressed as follows:
for some measure formula_36 and some family formula_48 of Hilbert spaces. The measure formula_36 is uniquely determined by formula_4 up to measure-theoretic equivalence; that is, any two measure associated to the same formula_4 have the same sets of measure zero. The dimensions of the Hilbert spaces formula_52 are uniquely determined by formula_4 up to a set of formula_36-measure zero.
The spaces formula_52 can be thought of as something like "eigenspaces" for formula_4. Note, however, that unless the one-element set formula_57 has positive measure, the space formula_52 is not actually a subspace of the direct integral. Thus, the formula_52's should be thought of as "generalized eigenspace"—that is, the elements of formula_52 are "eigenvectors" that do not actually belong to the Hilbert space.
Although both the multiplication-operator and direct integral formulations of the spectral theorem express a self-adjoint operator as unitarily equivalent to a multiplication operator, the direct integral approach is more canonical. First, the set over which the direct integral takes place (the spectrum of the operator) is canonical. Second, the function we are multiplying by is canonical in the direct-integral approach: Simply the function formula_61.
Cyclic vectors and simple spectrum.
A vector formula_23 is called a cyclic vector for formula_4 if the vectors formula_64 span a dense subspace of the Hilbert space. Suppose formula_4 is a bounded self-adjoint operator for which a cyclic vector exists. In that case, there is no distinction between the direct-integral and multiplication-operator formulations of the spectral theorem. Indeed, in that case, there is a measure formula_36 on the spectrum formula_27 of formula_4 such that formula_4 is unitarily equivalent to the "multiplication by formula_42" operator on formula_71. This result represents formula_4 simultaneously as a multiplication operator "and" as a direct integral, since formula_71 is just a direct integral in which each Hilbert space formula_52 is just formula_75.
Not every bounded self-adjoint operator admits a cyclic vector; indeed, by the uniqueness in the direct integral decomposition, this can occur only when all the formula_52's have dimension one. When this happens, we say that formula_4 has "simple spectrum" in the sense of spectral multiplicity theory. That is, a bounded self-adjoint operator that admits a cyclic vector should be thought of as the infinite-dimensional generalization of a self-adjoint matrix with distinct eigenvalues (i.e., each eigenvalue has multiplicity one).
Although not every formula_4 admits a cyclic vector, it is easy to see that we can decompose the Hilbert space as a direct sum of invariant subspaces on which formula_4 has a cyclic vector. This observation is the key to the proofs of the multiplication-operator and direct-integral forms of the spectral theorem.
Functional calculus.
One important application of the spectral theorem (in whatever form) is the idea of defining a functional calculus. That is, given a function formula_80 defined on the spectrum of formula_4, we wish to define an operator formula_82. If formula_80 is simply a positive power, formula_84, then formula_82 is just the formula_86 power of formula_4, formula_88. The interesting cases are where formula_80 is a nonpolynomial function such as a square root or an exponential. Either of the versions of the spectral theorem provides such a functional calculus. In the direct-integral version, for example, formula_82 acts as the "multiplication by formula_80" operator in the direct integral:
That is to say, each space formula_52 in the direct integral is a (generalized) eigenspace for formula_82 with eigenvalue formula_95.
General self-adjoint operators.
Many important linear operators which occur in analysis, such as differential operators, are unbounded. There is also a spectral theorem for self-adjoint operators that applies in these cases. To give an example, every constant-coefficient differential operator is unitarily equivalent to a multiplication operator. Indeed, the unitary operator that implements this equivalence is the Fourier transform; the multiplication operator is a type of Fourier multiplier.
In general, spectral theorem for self-adjoint operators may take several equivalent forms. Notably, all of the formulations given in the previous section for bounded self-adjoint operators—the projection-valued measure version, the multiplication-operator version, and the direct-integral version—continue to hold for unbounded self-adjoint operators, with small technical modifications to deal with domain issues.