Spectral Decomposition

The Central Idea

Suppose a linear operator has eigenvalues \(\lambda_1,\lambda_2,\ldots\) and corresponding mutually orthogonal spectral subspaces. Spectral decomposition expresses the operator as a sum of simple pieces, each associated with one spectral value.

In the cleanest finite-dimensional case,

where \(P_k\) is the orthogonal projector onto the eigenspace associated with \(\lambda_k\).

Each projector isolates one mode or eigenspace. The matrix or operator then acts by multiplying that component by its corresponding eigenvalue.

From Eigenvectors to Spectral Pieces

If \(v\) is an eigenvector of \(A\), then

This relation says that \(A\) acts very simply along the direction of \(v\). Spectral decomposition asks whether the whole space can be broken into directions or subspaces on which the operator acts this simply.

If it can, then every vector can be decomposed into spectral components, and the action of \(A\) can be understood component by component.

The Diagonalization Viewpoint

If a matrix \(A\) has a basis of eigenvectors, then it is diagonalizable. Writing the eigenvectors as the columns of a matrix \(V\), we have

where

The diagonal matrix \(\Lambda\) records the spectral values. In eigenvector coordinates, the transformation is just independent scalar multiplication along each mode.

Why Orthogonality Matters

Diagonalization is especially convenient when the eigenvectors can be chosen orthonormally. Let

have orthonormal columns. Then

so the decomposition becomes

For real matrices, \(Q^{*}=Q^T\). The orthonormal basis makes projections, reconstruction, norms, and numerical computation much more stable and transparent.

The Spectral Theorem for Real Symmetric Matrices

One of the most important spectral results says that every real symmetric matrix can be orthogonally diagonalized.

If

then there exists an orthogonal matrix \(Q\) and a real diagonal matrix \(\Lambda\) such that

The eigenvalues are real, and eigenvectors associated with distinct eigenvalues are orthogonal.

The Complex Version: Self-Adjoint Matrices

For complex matrices, the analogous condition is self-adjointness, also called Hermitian symmetry:

where \(A^{*}\) is the conjugate transpose.

A self-adjoint matrix has real eigenvalues and an orthonormal basis of eigenvectors, giving

This finite-dimensional theorem is the model for the spectral theory of self-adjoint differential and integral operators.

Normal Matrices

The orthonormal diagonalization principle extends beyond self-adjoint matrices. A complex matrix is normal when

Every normal matrix is unitarily diagonalizable:

Self-adjoint, unitary, and complex diagonal matrices are all important examples of normal matrices.

Spectral Projectors

Suppose \(q_k\) is a normalized eigenvector. The orthogonal projector onto its one-dimensional eigenspace is

Acting on a vector \(x\),

The scalar \(q_k^{*}x\) is the coordinate of \(x\) along the eigenvector \(q_k\). The projector extracts exactly that spectral component.

Reconstructing a Vector from Spectral Components

If \(q_1,\ldots,q_n\) form an orthonormal basis, then every vector \(x\) satisfies

Equivalently,

Therefore the projectors resolve the identity:

Reconstructing the Operator

Because \(A\) multiplies the \(k\)-th eigenspace by \(\lambda_k\),

Since the identity is the sum of the projectors,

Thus

This is the finite-dimensional spectral decomposition in projector form.

Repeated Eigenvalues

A repeated eigenvalue may have an eigenspace with dimension greater than one. In that case, it is often cleaner to use one projector for the entire eigenspace.

If \(E_{\lambda}\) is the eigenspace associated with \(\lambda\), let \(P_{\lambda}\) denote the orthogonal projector onto that space. Then

where the sum runs over distinct eigenvalues rather than individual eigenvectors.

Example: A Simple Symmetric Matrix

Consider

The eigenvalues are

Corresponding normalized eigenvectors are

Projectors for the Example

The projectors are

and

They satisfy

The spectral decomposition is therefore

Checking the Example

Substitute the projectors:

Adding gives

The original matrix is recovered.

How the Matrix Acts on a Vector

For any vector \(x\), write

Then

The component parallel to \(q_1\) is multiplied by \(3\), while the component parallel to \(q_2\) is multiplied by \(1\). The decomposition turns a coupled matrix transformation into independent modal scalings.

Powers Become Easy

Spectral decomposition makes repeated application of an operator especially simple. If

then the orthogonal projector relations imply

Therefore,

Example: A Matrix Power

For the matrix

we immediately obtain

In particular,

This avoids multiplying the matrix by itself four times.

Functions of a Matrix

The same idea works for many functions. If \(f\) is defined on the eigenvalues, then for a diagonalizable self-adjoint matrix we define

Instead of applying the function to every matrix entry, we apply it to the spectral values.

The Matrix Exponential

A fundamental example is the matrix exponential. If

then

This formula is central in linear differential equations and dynamical systems.

Linear Dynamical Systems

Consider

The solution is

Using spectral decomposition,

Every spectral component evolves independently according to its own exponential factor.

Growth, Decay, and Stability

The eigenvalues describe the long-term behavior of the spectral modes.

• If \(\lambda_k<0\), the mode \(e^{t\lambda_k}\) decays.
• If \(\lambda_k>0\), the mode grows.
• If \(\lambda_k=0\), the mode remains constant.

For more general complex eigenvalues, the real part controls exponential growth or decay while the imaginary part contributes oscillation.

Positive Semidefinite Matrices

A real symmetric matrix \(A\) is positive semidefinite when

for every vector \(x\).

Spectrally, this condition is equivalent to requiring

for every eigenvalue. This equivalence is important in optimization, covariance analysis, energy methods, and numerical linear algebra.

Quadratic Forms in Spectral Coordinates

If \(A=Q\Lambda Q^T\) and \(y=Q^Tx\), then

Therefore,

The quadratic form becomes a weighted sum of independent squared spectral coordinates.

The Rayleigh Quotient

For a nonzero vector \(x\), the Rayleigh quotient is

For a self-adjoint matrix, the Rayleigh quotient is bounded by the smallest and largest eigenvalues:

When \(x\) is an eigenvector, the quotient equals its eigenvalue.

Spectral Radius

The spectral radius of a matrix is

It measures the largest eigenvalue magnitude. In iterative methods and repeated dynamics, this quantity often determines whether powers of a matrix grow, remain bounded, or decay.

Dominant Modes

Suppose

Then for large powers,

is usually dominated by the \(\lambda_1^mP_1\) term, provided the initial vector has a nonzero component in that eigenspace.

This idea underlies the power method for approximating dominant eigenvectors.

Spectral Decomposition and Covariance

A covariance matrix is symmetric and positive semidefinite. Therefore it admits an orthogonal spectral decomposition

The eigenvectors identify orthogonal directions of variation, while the corresponding eigenvalues measure how much variance lies in those directions.

This is the mathematical foundation of principal component analysis.

Spectral Decomposition and Graphs

Graph adjacency matrices and graph Laplacians also have spectral structure. Their eigenvalues and eigenvectors encode information about connectivity, diffusion, clustering, and modes of variation across a network.

In particular, a graph Laplacian is symmetric for an undirected graph, so its spectral decomposition provides an orthogonal set of graph modes.

The Laplacian as a Spectral Operator

In continuous problems, differential operators play the role that matrices play in finite-dimensional linear algebra. For example, consider an eigenvalue problem for the negative second derivative:

together with suitable boundary conditions.

The eigenfunctions provide spectral modes, and the associated eigenvalues describe how strongly the operator acts on each mode.

From Matrices to Operators

In a finite-dimensional orthonormal eigenbasis,

For a self-adjoint operator with a purely discrete spectrum, the analogous form is

If each eigenspace is one-dimensional and generated by a normalized eigenfunction \(\phi_k\), then

Connection to Eigenfunction Expansions

In the preceding lesson, a function was expanded as

If

then

Spectral decomposition interprets this not merely as a convenient expansion of \(f\), but as a decomposition of the operator \(L\) itself into modal actions.

Discrete and Continuous Spectra

Finite matrices have finitely many eigenvalues. Differential and integral operators can have more complicated spectra.

• A discrete spectrum consists of isolated spectral values, often associated with ordinary eigenfunctions.
• A continuous spectrum contains a continuum of spectral values and is often represented by integrals rather than sums.
• Some operators contain both discrete and continuous spectral parts.

This distinction helps unify Fourier series with Fourier transforms.

Fourier Series as a Discrete Spectral Decomposition

On a bounded interval with periodic boundary conditions, complex exponentials

form discrete spectral modes for differentiation operators. A Fourier series therefore has the form of a discrete spectral expansion:

Each integer \(n\) labels one spectral mode.

Fourier Transforms as a Continuous Spectral Decomposition

On the whole real line, frequency is not restricted to integer values. Instead, the spectral variable \(\omega\) varies continuously.

The inverse Fourier transform reconstructs the function by integrating over the continuous spectrum:

The sum over discrete modes has become an integral over a continuum of modes.

The Spectral Measure Viewpoint

At a more advanced level, discrete sums and continuous integrals can be written in one common form. A self-adjoint operator can be represented formally as

where \(E(\lambda)\) represents a projection-valued spectral measure.

This notation generalizes the finite-dimensional formula

The full theory belongs to functional analysis, but the organizing idea is already familiar: resolve the space into spectral pieces, then let the operator multiply each piece by its spectral value.

Functional Calculus for Operators

Once an operator has been spectrally decomposed, functions of that operator can be defined spectrally. Formally,

In the discrete case this reduces to

This framework gives systematic meaning to objects such as \(e^{tL}\), \(L^{1/2}\), and resolvents where defined.

Spectral Decomposition of Diffusion

Suppose an operator \(L\) has eigenfunctions \(\phi_k\) satisfying

For a diffusion-type equation

each spectral component evolves as

Thus higher spectral values often correspond to modes that decay more rapidly.

Spectral Decomposition of Oscillation

For a wave-like equation

a mode associated with \(\lambda_k>0\) evolves according to an oscillator with angular frequency

The spectral values therefore encode the natural modal frequencies of the system.

Spectral Filtering

If a signal or state is decomposed into spectral components, we can modify those components selectively. Suppose

A spectral filter may produce

where \(H(\lambda)\) is a chosen response function. Some spectral components may be preserved, attenuated, amplified, or removed.

This viewpoint leads directly into filtering and signal analysis.

Low-Pass Behavior as a Spectral Operation

In many systems, rapidly varying modes are associated with larger spectral values. A low-pass or smoothing operation suppresses those high-frequency or high-spectral components while retaining slower modes.

Spectral decomposition makes this operation explicit: identify the modes, multiply their coefficients by the desired spectral response, and reconstruct the result.

Resolvents

Another important spectral object is the resolvent

where it exists. If

then

The expression fails when \(z\) reaches a spectral value because one denominator becomes zero. This is one reason the spectrum is closely connected with invertibility.

Spectrum and Invertibility

For a finite matrix, \(0\) belongs to the spectrum exactly when the matrix is not invertible.

If all eigenvalues are nonzero, then

Thus

Inverting the operator becomes reciprocal scaling of each spectral component.

Conditioning and Small Eigenvalues

When a self-adjoint positive definite matrix has a very small positive eigenvalue, inversion multiplies the corresponding spectral component by a very large number.

For positive definite \(A\), the spectral condition number in the Euclidean norm is

A large ratio signals sensitivity to perturbations and potential numerical difficulty.

Truncated Spectral Approximations

In large or infinite-dimensional systems, we may keep only selected spectral components. If the modes are ordered by importance, a truncated approximation is

The goal is to preserve the dominant structure while reducing computational complexity.

This idea appears in reduced-order models, data compression, principal component methods, and numerical simulation.

Spectral Decomposition Is Not Always Available in the Same Form

Not every matrix has an orthonormal eigenbasis. Some matrices are not even diagonalizable.

For example, a defective matrix may require Jordan form rather than an ordinary eigenvector decomposition. In infinite-dimensional settings, continuous spectrum can also prevent a simple sum over ordinary eigenfunctions.

The spectral theorem gives especially strong conclusions under symmetry, self-adjointness, or normality assumptions.

Spectral Decomposition versus Singular Value Decomposition

Spectral decomposition and singular value decomposition are related but distinct.

For a self-adjoint matrix,

uses eigenvalues, which may be positive, negative, or zero.

A general matrix has a singular value decomposition

where the singular values in \(\Sigma\) are nonnegative. SVD applies far more broadly, while spectral decomposition more directly describes how a square operator acts on its own invariant modes.

A Reliable Procedure for Symmetric Matrices

1. Verify that the matrix is symmetric or self-adjoint.
2. Find the eigenvalues.
3. Find a basis for each eigenspace.
4. Choose orthonormal eigenvectors within each eigenspace.
5. Form the matrix \(Q\) from those eigenvectors.
6. Form \(\Lambda\) from the corresponding eigenvalues.
7. Write \(A=Q\Lambda Q^{*}\).
8. Optionally form spectral projectors \(P_k=q_kq_k^{*}\).
9. Check \(\sum_kP_k=I\).
10. Check \(A=\sum_k\lambda_kP_k\).

Example: Decomposing a Vector Spectrally

Use the earlier matrix with

Let

The spectral coordinates are

Therefore,

Example: Applying the Matrix Spectrally

Since the corresponding eigenvalues are \(3\) and \(1\),

Thus

Converting back to standard coordinates gives

Direct multiplication gives the same result.

Example: Computing an Exponential

For

the exponential is

Substituting the projectors gives

Example: Solving a Linear System

Suppose

and \(A\) is invertible with spectral decomposition

Then

Each spectral component of the right-hand side is divided by the corresponding eigenvalue.

Common Error: Treating Any Matrix as Orthogonally Diagonalizable

The representation

with unitary \(Q\) is guaranteed for normal matrices, including real symmetric and complex self-adjoint matrices. It is not valid for every square matrix.

Always check the structural assumptions before using the spectral theorem.

Common Error: Confusing Eigenvalues with Singular Values

Eigenvalues can be negative or complex. Singular values are always real and nonnegative.

For a symmetric positive semidefinite matrix, the singular values coincide with the eigenvalues, but this agreement is a special case.

Common Error: Forgetting Normalization

The projector formula

assumes \(q\) is normalized so that \(q^{*}q=1\).

For a nonzero but unnormalized vector \(v\), the projector is

Common Error: Ignoring Repeated Eigenvalues

A repeated eigenvalue may correspond to several independent eigenvectors. The spectral projector belongs to the entire eigenspace, not to one arbitrarily chosen direction inside it.

Within a self-adjoint eigenspace, choose any orthonormal basis and sum the one-dimensional projectors to obtain the projector onto the full eigenspace.

Common Error: Assuming the Largest Eigenvalue Always Dominates

Dominance depends on the operation being performed and on the initial spectral coefficients.

For matrix powers, eigenvalue magnitude matters. For \(e^{tA}\), real parts matter. A mode cannot dominate a particular vector if that vector has zero projection onto the associated eigenspace.

Common Error: Using an Inverse When a Spectral Value Is Zero

If \(\lambda_k=0\), then the factor \(1/\lambda_k\) is undefined. A zero spectral value signals a nontrivial null space and prevents ordinary inversion.

In such cases, solvability, compatibility, or generalized inverses may need to be considered instead.

Practice 1: Identify the Spectral Pieces

Suppose

State the two eigenvalues represented by this decomposition.

Practice 2: Compute a Power

Using

find \(A^3\).

Practice 3: Compute a Matrix Function

If

write the positive spectral square root \(A^{1/2}\).

Practice 4: Inverse from the Spectrum

If

write \(A^{-1}\).

Practice 5: Matrix Exponential

If

write \(e^{tA}\).

Practice 6: Positive Semidefinite or Not?

A real symmetric matrix has eigenvalues

Is it positive semidefinite? Is it positive definite?

Practice 7: Spectral Radius

Find the spectral radius of a matrix whose eigenvalues are

Practice 8: Projector from a Normalized Eigenvector

Let

Find the orthogonal projector \(P=qq^T\).

Practice 9: Modal Evolution

Suppose

where \(q_1,q_2\) are orthonormal eigenvectors of \(A\) with eigenvalues \(-1\) and \(-4\). Solve

Practice 10: Explain the Operator Advantage

Suppose a self-adjoint operator has orthonormal eigenfunctions \(\phi_n\) with

If

explain why spectral coordinates make applying \(L\) easier.

Practice Answers

Practice 1

The spectral coefficients multiplying the projectors are the eigenvalues:

Practice 2

Powers act on each spectral value independently:

Therefore,

Practice 3

Apply the positive square-root function to the eigenvalues:

Practice 4

Take reciprocals of the nonzero eigenvalues:

Practice 5

Apply the exponential to each eigenvalue:

Practice 6

All eigenvalues are nonnegative, so the matrix is positive semidefinite.

It is not positive definite because one eigenvalue equals zero.

Practice 7

The spectral radius is the largest eigenvalue magnitude:

Therefore,

Practice 8

Compute the outer product:

Thus

Practice 9

Each initial spectral coefficient evolves exponentially according to its eigenvalue:

Practice 10

Applying \(L\) gives

Therefore,

The operator no longer mixes the modes. It simply multiplies each spectral coefficient by its corresponding eigenvalue.

What to Remember

The core finite-dimensional spectral representation is

For an orthonormal eigenbasis, the same decomposition can be written

• eigenvalues identify the spectral values of an operator;
• eigenvectors or eigenfunctions identify its natural modes;
• spectral projectors isolate individual eigenspaces;
• the projectors resolve the identity when the modes are complete;
• self-adjoint and normal matrices admit orthonormal spectral bases;
• matrix powers become powers of eigenvalues;
• matrix functions become ordinary functions of eigenvalues;
• linear dynamics evolve independently in spectral coordinates;
• positive semidefiniteness is equivalent to nonnegative eigenvalues for self-adjoint matrices;
• small eigenvalues can make inversion sensitive;
• truncated spectra provide reduced approximations;
• Fourier series illustrate discrete spectral decomposition;
• Fourier transforms illustrate continuous spectral decomposition;
• eigenfunction expansions are the function-space version of the same idea;
• spectral filtering modifies selected modes by changing their spectral weights.

The most useful perspective is that spectral decomposition chooses coordinates in which a linear operator becomes as close as possible to independent scalar actions.

Where We Go Next

Spectral decomposition gives us a mode-by-mode description of signals, matrices, and operators. Once those modes are visible, we can deliberately change their weights to suppress noise, preserve desired frequencies, isolate bands, smooth data, or emphasize selected behavior.

The next lesson, Filtering and Signal Analysis, develops that operational viewpoint. We will connect frequency response, low-pass and high-pass filters, band selection, convolution kernels, transfer functions, spectral attenuation, noise reduction, and practical signal interpretation.