Return to computing page for the first course APMA0330
Return to computing page for the second course APMA0340
Return to computing page for the fourth course APMA0340
Return to Mathematica tutorial for the first course APMA0330
Return to Mathematica tutorial for the second course APMA0340
Return to Mathematica tutorial for the fourth course APMA0360
Return to the main page for the first course APMA0330
Return to the main page for the second course APMA0340
Return to the main page for the fourth course APMA0360
Return to Part V of the course APMA0340
Introduction to Linear Algebra with Mathematica
Glossary
Preface & Intuition
A typical Sturm–Liouville problem for a second-order self-adjoint differential operator (referred to as the Sturm–Liouville operator)
consists of a differential equation on a finite interval (𝑎`, b) containing an eigenvalue parameter λ:
or equivalently expressed as:
subject to separated boundary conditions:
Here, p(x) ≥ 0, q(x), and w(x) > 0 are continuous on the closed interval [𝑎, b], and p(x) possesses a continuous derivative. The positive function w(x) is called the weight or density function. While boundary conditions of the third kind \eqref{EqOrtho.2} are chosen here for illustration, other types may be used (for instance, classical Fourier series expansions rely on periodic boundary conditions). Crucially, any valid boundary conditions must satisfy Lagrange identity, which guarantees that the resulting linear differential operator is self-adjoint.
The objective is to find the values of λ (real or complex) for which this boundary value problem (BVP) admits non-trivial solutions (not indentically zero). These specific parameters are called eigenvalues, and their corresponding non-zero solutions are called eigenfunctions. When the structural parameters of the equation are non-negative, the operator \(L[x, \texttt{D}]\) is non-negative, yielding entirely non-negative eigenvalues.
When the method of separation of variables is applied to linear partial differential equations (PDEs), it naturally reduces the system to a Sturm–Liouville framework. Solving the PDE then hinges on a second, equally critical step: representing an arbitrary function as an infinite series expansion of these geometric eigenfunctions. For a vast class of second-order PDEs, the generated eigenfunctions form an orthogonal, complete set that makes explicit calculations possible.
Assuming the problem conditions guarantee a discrete sequence of eigenvalues 0 ≤ λ₀ < λ₁ < λ₂ < ··· → ∞, we can map out a given function f(x) over the interval [𝑎, 𝑏] using a linear combination of the corresponding eigenfunctions ϕₙ:
To ensure convergence at the endpoints, we assume that f(x) matches the underlying homogeneous boundary conditions \eqref{EqOrtho.2}. The definition of "approximation" can take several forms. Ideally, we want the coefficients cₙ chosen so that the finite partial sum,
While pointwise convergence is ultimately required to evaluate exact physical solutions constructed via separation of variables, it is mathematically far simpler to initially calculate coefficients cₙ such that Sₙ(f; x) minimizes errors globally. This framework introduces us to approximation in the least-squares (or mean) sense.
Inner Products & Orthogonality
Let us consider a set X of real- or complex-valued functions on a finite interval [𝑎, 𝑏]. For a positive weight function w(x) and two arbitrary functions f and g from X, we can define an inner product by:
where the asterisk (\(^{\ast}\)) or overline (\(\overline{f}\)) denotes the complex conjugate operation. Note that it is common to denote the inner product either with a vertical bar (standard in physics, Dirac bra-ket notation) or a comma (standard in mathematics). For instance, if \( \ f(x) = u(x) + {\bf j} v(x) \ \) is a complex-valued function, then \( \ \overline{f(x)} = f^{\ast} (x) = u(x) - {\bf j} v(x) \ \) represents its complex conjugate.
With this newly introduced inner product and its corresponding norm, we can treat the functions in our space geometrically as if they were vectors, establishing a clear analogy with classical Euclidean spaces.
Definition (The Norm): If f(x) is a square-integrable function defined on an interval [𝑎, 𝑏], we define the square norm of f to be:
The set of all functions on the interval [𝑎, 𝑏] having a finite norm \eqref{EqOrtho.6} is denoted by 𝔏²([𝑎, 𝑏], w) or simply 𝔏².
Recall that the definite integral of a function over an interval \([a, b]\) does not depend on the specific values of the function at a discrete number of points because isolated points carry a measure of zero and do not alter the calculation of the area. Therefore, when computing an integral, it does not matter what values the function takes at the boundary endpoints, or even whether it is explicitly defined there. When we wish to remain neutral about endpoint inclusion, we deploy a convenient notation: |𝑎, 𝑏|, which can represent any of the following open, closed, or half-open intervals:
Definition (Orthogonality): Two functions f(x) and g(x) from 𝔏² defined on some interval |𝑎, 𝑏| are called orthogonal if the inner product of the two functions over that interval is zero:
This perpendicular metric property of functions is abbreviated as \(f \perp g\).
To guarantee the analytical completeness of 𝔏²([𝑎, 𝑏], w), we must utilize Lebesgue integration in the definition of our norm rather than standard Riemann integration. Generally speaking, 𝔏²([𝑎, 𝑏], w) represents the topological completion of this functional vector space. Equipped with this inner product and complete metric, the set of functions 𝔏² transforms into a functional Hilbert space.
𝔏²([𝑎, 𝑏], w) is a classic example of an infinite-dimensional linear vector space, meaning its elements cannot be constructed using linear combinations of a restricted, finite number of basis functions. Because expansions like Eq. \eqref{EqOrtho.3} require an infinite sum, several vital issues arise: When does this series converge? Does a converged sum always recover the exact starting function?
Definition (Orthogonal System): A collection of functions { ϕ₀(x), ϕ₁(x), … , ϕₙ(x), … } defined on |𝑎, 𝑏| is called an orthogonal system on |𝑎, 𝑏| if:
Example 1: The Rademacher system, named after Hans Rademacher (1892--1969), is an orthogonal sequence of square-wave functions \[ r_n (x) = \mbox{sgn}\left( 2^n \pi x \right) , \qquad n \in \mathbb{N} = 0,1,2,\ldots, \] defined on the unit interval [0, 1]. Key attributes include being an incomplete orthonormal system, stochastically independent, and forming the foundation for the complete Walsh system.
Each Rademacher function alternates signs between −1 and +1 over intervals of length 2−(n+1). The n-th Rademacher function can also be defined as \[ r_n (t) = (-1)^{\left\lfloor 2^n t \right\rfloor} , \qquad t \in [0, 1] . \] In particular,
- r₀(𝑥) changes sign once at 𝑥 = ½,
- r₁(𝑥) changes sign at 𝑥 = ¼, ½, ¾,
- r₂(𝑥) changes sign every ⅛.
- The Rademacher functions are orthogonal in 𝔏²([0, 1]): \[ \langle r_m \mid r_n \rangle = \int_0^1 r_m (x)\,r_n (x)\,{\text d}x = \delta_{m,n} = \begin{cases} 1, \qquad& m=n , \\ 0, \qquad& m\ne n . \end{cases} \]
- They are normalized: ∥r∥₂ = 1.
- They are independent random variables when [0, 1] is equipped with Lebesgue measure.
- The Rademacher system is incomplete.
To prove that the Rademacher system is not a complete basis (not a full basis) for 𝔏²([0,1]), we must find a non-zero function 𝑓 ∈ 𝔏²([0,1]) that is orthogonal to every Rademacher function rₙ(𝑡) for all n = 0, 1, 2, …. If the system were complete, the only function orthogonal to every element would be the zero function.
The Explicit Counterexample Function: Define the function 𝑓(𝑡) on the interval [0, 1] as the product of the first two Rademacher functions: \[ f(t) = r_0 (t) \cdot r_1 (t) . \] Visualizing the function, we get
- r₀ is +1 on (0., 0.5) and −1 on (0.5, 1).
- r₁ is +1 on (0., 0.25) ∪ (0.5, 0.75) and −1 on (0.25, 0.5) ∪ (0.75, 1).
- Multiplying them yields 𝑓(𝑡), which takes the following values: \[ f(t) = \begin{cases} +1 , \qquad& t \in (0, 0.25), \\ -1 , \qquad& t \in (0.25, 0.5) , \\ -1 , \qquad& t \in (0.5, 0.75) , \\ +1 , \qquad& t \in (0.75, 1.) . \end{cases} \]
Proving Orthogonality of 𝑓(𝑡) to r₀(𝑡) and r₁(𝑡). We compute the inner product \( \displaystyle \quad \left\langle f \mid r_n \right\rangle = \int_0^1 f(t)\,r_n (t)\,{\text d}t \ \) for the first two cases:
- For n = 0: \[ \left\langle f , r_0 \right\rangle = \int_0^1 \,\left( r_0 (t) \cdot r_1 (t) \right) r_0 (t)\,{\text d}t = \int_0^1 \,\left( r_0 (t) \right)^2 \,r_1 (t) \,{\text d}t . \] Since \( \displaystyle \quad \left( r_0 (t) \right)^2 = 1 \ \) almost everywhere, we get \[ \left\langle f , r_0 \right\rangle = \int_0^1 \,r_1 (t)\,{\text d}t = 0. \]
- For n = 1: \[ \left\langle f , r_1 \right\rangle = \int_0^1 \,\left( r_0 (t) \cdot r_1 (t) \right) r_1 (t)\,{\text d}t = \int_0^1 \,\left( r_1 (t) \right)^2 \,r_0 (t) \,{\text d}t = \int_0^1 \,r_0 (t)\,{\text d}t = 0 . \]
Proving Orthogonality to Higher Rademacher Functions n ≥ 2. For any n ≥ 2, the function rₙ(𝑡) oscillates much faster than 𝑓(𝑡). We look at the inner product: \[ \left\langle f \mid r_n \right\rangle = \int_0^1 \,r_0 (t) \,r_1 (t) \,r_n (t)\,{\text d} t . \] Because Rademacher functions are stochastically independent random variables with an expected value (integral) of 0, the integral of their product equals the product of their integrals: \[ \int_0^1 \,r_0 (t) \,r_1 (t) \,r_n (t)\,{\text d} t = \left( \int_0^1 \,r_0 (t) \,{\text d} t \right) \cdot \left( \int_0^1 \,r_1 (t) \,{\text d} t \right) \cdot \left( \int_0^1 \,r_n (t) \,{\text d} t \right) = 0 \cdot 0 \cdot 0 = 0 . \] Alternatively, you can observe that on each of the four quadrants of length 0.25 where 𝑓(𝑡) is constant (+1 or −1), the function rₙ(𝑡) completes one or more full symmetric square-wave cycles. The integral of over each quadrant is exactly 0, causing the total integral to vanish.
This definitively proves that the Rademacher system lacks the completeness required to form a full basis for 𝔏².
Now we show that Rademacher functions are stochastically independent random variables with an expected value (integral) of 0.
Expected Value (Integral) is 0. For any Rademacher function , the function takes the value on exactly half of the interval [0, 1] and −1 on the other half. \[ \int_0^1 \,r_n (t)\,{\text d}t = \left( 1 \times \frac{1}{2} \right) + \left( -1 \times \frac{1}{2} \right) = 0 . \] In probability terms, the expected value E[rₙ] = 0.
Stochastic Independence via Dyadic Intervals: To prove that r₀, r₁, …, rₙ are stochastically independent, we must show that for any choice of signs ε ₖ ∈ {−1, +1}, the measure (length) of the set where they simultaneously hold those values matches the product of their individual probabilities: \[ \mu \left( \left\{ t \in [0, 1]\ : \ r_0 (t) = \varepsilon_0 \cdot r_1 (t) = \varepsilon_1 , \ldots , r_n (t) = \varepsilon_n \right\} \right) = \frac{1}{2^{n+1}} . \] because each successive Rademacher function splits the previous subintervals exactly in half, choosing a sequence of signs (ε₀, ε₁, … , εₙ) pinpoints exactly one specific dyadic interval of length 1/2n+1. Since this matches \( \displaystyle \quad \prod_{k=0}^n\,\Pr [r_k = \varepsilon_k ] = (1/2)^{n+1} ,\quad \) the functions are mutually stochastically independent.
Rigorous Step for the Joint Integral of the inner product: For any measurable, independent random variables, the expectation of their product equals the product of their expectations (E[XYZ] = E[X] E[Y] E[Z]). Using the joint distribution (the previous step) to evaluate the integral for n ≥ 2: \begin{align*} \left\langle f(t) , r_n (t) \right\rangle &= \int_0^1 \,r_0 (t)\,r_1 (t)\,r_n (t) \,{\text d}t \\ &= \sum_{\varepsilon_0 , \varepsilon_1 , \varepsilon_n \in \{ 1, -1 \}} \ \left( \varepsilon_0 \cdot \varepsilon_1 \cdot \varepsilon_n \right) \cdot \mu \left( r_0 = \varepsilon_0 , \ r_1 = \varepsilon_1 , \ r_n = = \varepsilon_n \right) \\ &= \sum_{\varepsilon_0 , \varepsilon_1 , \varepsilon_n \in \{ 1, -1 \}} \ \left( \varepsilon_0 \cdot \varepsilon_1 \cdot \varepsilon_n \right) \cdot \left( \frac{1}{2}\cdot \frac{1}{2} \cdot \frac{1}{2} \right) \\ &= \left( \sum_{\varepsilon_0} \,\frac{\varepsilon_0}{2} \right) \cdot \left( \sum_{\varepsilon_1} \,\frac{\varepsilon_1}{2} \right) \cdot \left( \sum_{\varepsilon_n} \,\frac{\varepsilon_n}{2} \right) \\ &= \left( \int_0^1 \,r_0 (t)\,{\text d}t \right) \cdot \left( \int_0^1 \,r_1 (t)\,{\text d}t \right) \cdot \left( \int_0^1 \,r_n (t)\,{\text d}t \right) \\ &= 0\cdot 0 \cdot 0 = 0 . \end{align*} This clean separation completes the missing logical leap in the counterexample.
Summary Rademacher functions are independent because any intersection of their sign states yields a single dyadic interval of length 2−(n+1). Because they are independent and centered (E[rₖ] = 0), the integral of their product strictly separates into the product of their individual integrals, forcing ⟨ r₀r₁ ∣ rₙ ⟩ = 0. ■
Example 2: The Rademacher system is closely related to the Walsh system. In fact, the Walsh functions are obtained as finite products of Rademacher functions. If \[ n = \sum_{k\ge 0} \varepsilon_k\,2^k , \qquad \varepsilon_k \in \{ 0, 1 \} , \] is the binary expansion of the integer n, then the n-th Walsh function is \[ w_n (x) = \prod_{k\ge 0} \,r_k (x)^{\varepsilon_k} , \] where only finitely many factors differ from 1.
Khintchine's inequality states that for any sequence of real coefficients (𝑎₁, 𝑎₂, … , 𝑎ₙ) ∈ ℝⁿ and any 0 < p ≤ ∞, the norm of a linear combination of Rademacher functions is strictly equivalent to its 𝔏² norm. Spesifically, there exist positive constants Ap and Bp depending only on p such that \[ A_p \left( \sum_{i=1}^n \ a_i^2 \right)^{1/2} \le \left( \int_0^1 \left\vert \sum_{i=1}^n a_i r_i (t) \right\vert^p \,{\text d}t\right)^{1/p} \le B_p \left( \sum_{i=1}^n \ a_i^2 \right)^{1/2} . \] This inequality shows that for Rademacher series, all topologies coincide. This is highly unusual for general functional series, where higher moments typically grow much faster. Therefore, the Rademacher functions mimic independent fair coin flips, this inequality bounds the absolute moments of random walks and serves as a pillar in functional analysis and Banach space theory.
The Rademacher system is incomplete because it cannot represent functions that lack symmetry across dyadic intervals. To fix this, Raymond Paley (1907--1933) introduced a method to complete the system by taking all possible finite products of Rademacher functions, forming the complete orthonormal Walsh system.
The Paley Ordering Rule
To construct the m-th Walsh function in Paley order, look at the binary expansion of the integer m: \[ m = \sum_{k=0}^j \ b_k 2^k , \qquad b_k \in \{ 0, 1 \} . \] The function wm(t) is then defined as \[ w_m (t) = \prod_{k=0}^j \ \left( r_k (t) \right)^{b_k} . \] Step-by-Step Generation Example
- w₀(𝑡): Binary representation is 0. The empty product equals 1.
- w₁(𝑡): Binary is 1 = 1·2⁰. Thus, w₁(𝑡) = r₀(𝑡).
- w₂(𝑡): Binary is 2 = 1·2¹ + 0·2⁰. Thus, w₂(𝑡) = r₁(𝑡).
- w₃(𝑡): Binary is 3 = 1·2¹ + 1·2⁰. Thus, w₃(𝑡) = r₁(𝑡) · r₀(𝑡).
Least Squares Approximation
Let us denote by 𝔏²([𝑎, b], w) the space of real or complex-valued functions f(x) having a finite weighted 𝔏² norm:
Approximation in the least squares sense means that the norm of the deviation between the target function and its finite partial sum Sₙ(f; x) approaches zero as n → ∞:
Crucially, the vanishing of this integral does not imply that Sₙ(f; x) becomes close to f(x) at every single point x. Rather, it guarantees that \(S_N(f; x)\) converges to f(x) globally, allowing deviations only on sets of points whose total aggregate length is arbitrarily small. If
we say that the sequence Sₙ(f; x) converges in the mean (or mean-square sense) to f(x), which is usually abbreviated as f(x) = l.i.m.Sₙ(f; x).. This global error metric is formally called the mean square deviation.
We consider the problem of minimizing this deviation by approximating f(x) using a complete, orthogonal sequence of eigenfunctions { ϕₙ(x) }n≥0. We assume that these functions correspond to distinct eigenvalues of a regular Sturm–Liouville operator, making them strictly orthogonal on the interval [𝑎, 𝑏] with respect to the positive weight density w(x):
Derivation of the Optimal Coefficients
To find the best possible coefficients \(c_k\) that minimize the approximation error, we expand the squared norm of the error expression explicitly:
By completing the square for each variable cₖ within the complex numbers, this identity transforms into:
Notice that the chosen coefficients cₖ appear exclusively inside the first summation block. Because this term is a sum of non-negative squares, the entire global expression is minimized if and only if each independent term in that first sum is forced to equal zero. This forces our optimal coefficients to satisfy:
This reveals a powerful mathematical truth: the formula \eqref{EqOrtho.7} provides the absolutely best approximation to f(x) in the sense of least squares. Crucially, due to the underlying orthogonality of the eigenfunctions, each individual coefficient cₖ is completely independent of n (the total number of terms included in the partial sum).
The constants defined by \eqref{EqOrtho.7} are the generalized Fourier coefficients of the function f(x) with respect to the orthogonal system { ϕₖ(x) }, and the resulting infinite series is called the corresponding Fourier series. Because this series is not guaranteed to converge pointwise to the target function at every coordinate, we deploy a tilde notation (∼) to indicate an asymptotic mapping rather than a definitive pointwise identity:
This notation states that cₖ are the calculated Fourier coefficients, establishing a systematic analytical blueprint to recover the function f(x).
Alternative Derivation via Uniform Convergence
The formula \eqref{EqOrtho.7} can also be derived directly if we assume a priori that the expansion converges to f(x) uniformly across the domain. Under the safety of uniform convergence, one can multiply both sides of the series equation by ϕₙ(x)✶ w(x) and integrate term-by-term across the interval [𝑎, 𝑏]:
By invoking the orthogonality relation, every single integrated term in the infinite summation collapses to zero except for the single index where k = n, yielding:
which successfully reproduces our optimal coefficient equation \eqref{EqOrtho.7}.
Example 3: Let us consider a sequence of Zernike polynomials that are orthogonal over the area of a two-dimensional unit disk. Named after Dutch optical physicist Frits Zernike (1888–1966), laureate of the 1953 Nobel Prize in Physics and inventor of phase-contrast microscopy, these polynomials are indispensable in advanced beam optics, imaging, and ophthalmology for modeling wavefront aberrations.
Zernike polynomials map circular regions natively by separating coordinates into a radial component \(r\) and an azimuthal angle \(\theta\). They are split into angularly even and odd functions. For non-negative integers satisfying \(n \ge m \ge 0\) where the difference \(n-m\) is an even number, the odd Zernike polynomials are defined as:
and the corresponding even Zernike polynomials are defined as:
Here, the variables are bounded within polar parameters \(0 \le r \le 1\) and \(0 \le \theta < 2\pi\). The term \(R_n^m(r)\) represents the highly oscillatory radial polynomial mapping, defined explicitly by the combinatorial sum: \[ R_n^m (r) = \sum_{k=0}^{\frac{n-m}{2}} \frac{(-1)^k \, (n-k)!}{k! \left(\frac{n+m}{2}-k\right)! \left(\frac{n-m}{2}-k\right)!} \, r^{n-2k} . \]
To align with our least squares framework, the inner product over the two-dimensional circular domain contains an explicit area element \({\text d}A = r\,{\text d}r\,{\text d}\theta\), indicating that the radial variable \(r\) itself acts as the internal weight density function. The double integral orthogonality relationship over the entire unit disk evaluates as follows:
where δ is the Kronecker delta, and \(\epsilon_m\) is the Neumann factor (evaluating to \(2\) if \(m=0\), and \(1\) otherwise). This strict 2D orthogonality means any distorted optical phase wavefront can be reconstructed in the least-squares sense with absolutely zero informational overlap between its expansion coefficients.
Zernike polynomials are orthogonal over a 2D unit disk and are widely used in optics to model wavefront aberrations. They separate coordinates into a radial component \(r\) and an azimuthal angle \(\theta\), categorized into even and odd functions defined via radial polynomials \(R_n^m(r)\).
Explicit Low-Order Zernike Polynomial Formulas
The following table summarizes the explicit mathematical formulas for lower-order unnormalized Zernike polynomials \(Z_n^{\pm m}(r, \theta)\) and their corresponding optical aberration names:
| Radial (\(n\)) | Azimuthal (\(m\)) | Notation | Explicit Formula | Classical Name | 0 | 0 | \(Z_0^0(r, \theta)\) | \(1\) | Piston |
|---|---|---|---|---|
| 1 | -1 | \(Z_1^{-1}(r, \theta)\) | \(r \sin \theta\) | Vertical Tilt (Y-Tilt) |
| 1 | 1 | \(Z_1^1(r, \theta)\) | \(r \cos \theta\) | Horizontal Tilt (X-Tilt) |
| 2 | -2 | \(Z_2^{-2}(r, \theta)\) | \(r^2 \sin 2\theta\) | Oblique Astigmatism |
| 2 | 0 | \(Z_2^0(r, \theta)\) | \(2r^2 - 1\) | Defocus |
| 2 | 2 | \(Z_2^2(r, \theta)\) | \(r^2 \cos 2\theta\) | Vertical Astigmatism |
| 3 | -1 | \(Z_3^{-1}(r, \theta)\) | \((3r^3 - 2r)\sin \theta\) | Vertical Coma |
| 3 | 1 | \(Z_3^1(r, \theta)\) | \((3r^3 - 2r)\cos \theta\) | Horizontal Coma |
| 4 | 0 | \(Z_4^0(r, \theta)\) | \(6r^4 - 6r^2 + 1\) | Primary Spherical Aberration |
The inner product and orthogonality properties over the circular domain allow wavefronts to be reconstructed accurately using least-squares expansion coefficients.
■Example 4: In introductory textbooks, least squares problems typically involve smooth, symmetric functions with clean, closed-form analytic solutions. However, in physical applications—such as localized atmospheric thermal turbulence or complex structural lens defects—the target function is often highly non-linear, off-center, and analytically non-integrable. In these cases, numerical computer calculations are strictly essential to evaluate the generalized Fourier coefficients.
Let us consider an asymmetrical, off-center thermal bloom pocket modeled as a localized Gaussian anomaly superimposed on a radial cubic deformation over the unit circular aperture: \[ f(r, \theta) = 3.5 \exp\left\{ -8 \left[ (r\cos\theta - 0.35)^2 + (r\sin\theta + 0.25)^2 \right] \right\} - 1.2r^3 . \]
Because of the shifted coordinates inside the exponential term, computing the projection inner product integrals \(\langle f, Z_n^{\pm m} \rangle\) analytically is impossible. We must deploy 2D numerical quadrature (numerical integration) to extract the best approximation coefficients in the mean-square sense.
You can execute the following complete, self-contained Mathematica script to numerically compute the generalized Fourier-Zernike coefficients up to radial degree \(N=4\) via adaptive 2D quadrature and visualize the structural minimization of the mean square deviation.
|
|
Pedagogical Elements:
-
The Role of the Jacobian Coordinate Weight: Inside the
NIntegrate[]function loop, the expression explicitly contains* rmatching the polar element metric rdr&thicksp;dθ. This demonstrates how polar space inherently alters the internal geometric weight density. -
Mean Convergence vs. Local Over-smoothing: Students will observe in the generated
errorPlotthat while global least-squares error metrics approach zero, localized spike errors persist near the coordinate point \((0.35, -0.25)\). Truncating at low radial frequencies (\(N=4\)) naturally smooths out steep structural gradients, highlighting why higher modal bounds are required to capture point-wise spatial variations.
Bessel's Inequality
Named after the prominent German astronomer and mathematician Friedrich Wilhelm Bessel (1784–1846), the following fundamental inequality was first formulated in 1828 during his investigations into planetary perturbations and Keplerian orbital math.
While its technical definition can appear abstractly dry, its geometric meaning is beautifully simple: it represents the infinite-dimensional analogue of the Pythagorean theorem. Just as the sum of the squared projections of a 3D vector onto two axes cannot exceed the total square length of the vector itself, the total aggregate energy contained within a finite set of coordinate components can never break the absolute energy barrier of the original function. It guarantees that our series approximations remain physically stable and bounded.
Theorem 1 (Bessel's Inequality): Let ℌ be an inner product space or Hilbert space, and let { e₁, e₂, … , eₙ, …} be a given orthonormal sequence. Then for any element f from ℌ, Bessel's inequality states:
where ⟨· , ·⟩ denotes the standard inner product in the space.
By leveraging the sesquilinear properties of the inner product space, this expands into:
Since our basis sequence is strictly orthonormal, the cross-product components collapse under the Kronecker delta property ⟨ eₖ , e𝓂 ⟩ = δk,m. Substituting \(c_k = \langle \mathbf{e}_k, f \rangle\) and \(\overline{c_j} = \langle f, \mathbf{e}_j \rangle\), the relationship simplifies directly to:
Rearranging the remaining elements yields the inequality for any finite index bound \(n\):
Since this bounding upper constraint holds true for every finite \(n\), taking the limit as n → ∞ successfully establishes Bessel's global inequality statement.
Corollary 1 (Classical Trigonometric framework):
Then the corresponding bounding total energy relationship satisfies:
where \(E[f]\) represents the global structural energy of the \(2\ell\)-periodic function.
Example 5: To demonstrate Bessel's inequality to students dynamically, we choose a classic piecewise discontinuous sawtooth wave \(f(x) = x\) over the periodic bounds \([-\pi, \pi]\) with a system scale \(\ell = \pi\). The global energy integral evaluates explicitly as: \[ E[f] = \frac{1}{\pi} \int_{-\pi}^{\pi} x^2 \,{\text d}x = \frac{2\pi^2}{3} \approx 6.57973 . \]
Because our function is strictly an odd function, the cosine components vanish instantly (\(a_n = 0\)), while integrating the sine projections yields the sequence \(b_n = \frac{2(-1)^{n+1}}{n}\). Truncating our infinite summation to a restricted integer limit \(N\) changes the total energy identity into a strict unequal boundary: \[ \sum_{n=1}^N b_n^2 = \sum_{n=1}^N \frac{4}{n^2} < \frac{2\pi^2}{3} . \]
You can execute the following code to numerically track how the sum of the projection squares steps up asymptotically toward the global energy ceiling without ever breaking it, proving Bessel's constraint visually:
Dense Sets, Completeness, and Bases
Recall that the closure \( \displaystyle \ \overline{A}\ \) of a subset A ⊆ ℌ is the smallest closed set containing A. It consists of all limits of convergent sequences (xₙ) where each xₙ ∈ A.
In other words, a subset A ⊆ ℌ is dense in a Hilbert space ℌ if for every f ∈ ℌ and every ε > 0, there exists g ∈ A with
For instance, in the Hilbert space 𝔏²([0,2π]) with inner product
Example 6: To help you master the topological definition of density in infinite-dimensional spaces, let us analyze the standard sequence space ℓ², which consists of all real sequences x = (x₁, x₂, x₃, &hellp;) such that \(\|x\|^2 = \sum_{i=1}^\infty x_i^2 < \infty\).
We define two subsets, A and B, which both look "infinitely large" and span across all coordinate directions, yet have radically different density states.
Case A: The Space of Finite Sequences (Finitely Terminating Sparsity)
Let A be the set of all sequences that have only a finite number of non-zero terms, becoming exactly zero after some index N (where N varies for each sequence): \[ A = \left\{ \mathbf{x} \in \ell^2 \;\middle|\ \exists N \in \mathbb{N} \text{ such that } x_k = 0 \text{ for all } k > N \right\} . \]
Is A dense in ℓ²? YES.
Proof sketch for students: Take an arbitrary vector f = (f₁, f₂, f₃, …) ∈ ℓ² and pick any ε > 0. Because the infinite series \(\sum_{k=1}^\infty f_k^2\) converges, the "tail" of the series must vanish as we go out to infinity. This means we can always find a truncation index N large enough such that \(\sum_{k=N+1}^\infty f_k^2 < \varepsilon^2\).
If we build a vector \( g \in A\) by keeping the first N terms of f and setting the rest to zero (\(g = (f_1, f_2, \dots, f_N, 0, 0, \dots)\)), then: \[ \|f - g\|^2 = \sum_{k=N+1}^\infty f_k^2 < \varepsilon^2 \quad\implies \quad \|f - g\| < \varepsilon . \] Because we can get arbitrarily close to any sequence in the space by truncating its tail, **\(A\) is dense in \(\ell^2\)**.
Case B: The Standard Hilbert Cube Basis Set (Orthonormal Coordinate Matrix)
Now let B be the set containing exclusively the standard orthonormal basis vectors \(e_1, e_2, e_3, \dots\), where each vector contains a single \(1\) and zeros everywhere else: \[ B = \left\{ e_1, e_2, e_3, \dots \right\} \quad \text{where } e_1=(1,0,0,\dots), \; e_2=(0,1,0,\dots) . \]
Is \(B\) dense in ℓ²? NO.
Proof sketch for students: Students often assume that because B points down every dimension axis out to infinity, it must "fill" the space. However, let us calculate the distance between any two distinct basis vectors inside B: \[ \| e_i - e_j \| = \sqrt{1^2 + (-1)^2} = \sqrt{2} \quad (\text{for } i \neq j) . \] Every single point in B is isolated from every other point by an absolute distance of exactly \(\sqrt{2}\).
If we choose the zero vector \(f = (0, 0, 0, \dots) \in ℓ² and set our target accuracy to \(\varepsilon = 0.5\), we find that the distance from the zero vector to *any* point in \(B\) is exactly \(\|f - e_k\| = 1\). It is impossible to find a point in B that gets closer to the origin than a distance of \(1\). Because a massive open ball of radius \(1\) around the origin contains absolutely no elements of \(B\), **\(B\) is not dense in \(\ell^2\)**.
The Structural Takeaway for Students
This contrast highlights why linear combinations are the missing ingredient that turns a discrete collection into a space-filling system. While the basis set \(B\) itself is incredibly sparse and **not dense**, taking its finite linear span creates the set \(A\), which is completely **dense** in the Hilbert space!
This example shows that in infinite dimensions, sets can point in every single coordinate direction out to infinity without being dense, because the space has enough "room" to isolate infinite sets of points from one another. ■
Not every infinite collection of orthogonal functions is rich enough to expand every hidden function in 𝔏²([𝑎, b], w); this brings us directly to the concept of completeness. How do we generate suitable sets of orthogonal functions? How do we calculate the coefficients cₙ when given a function f and a set of basis functions \(\{\phi_n(x)\}\)? We will resolve these core questions sequentially throughout this section.
Definition (Complete / Total System): A complete system (or complete set) of vectors in a Hilbert space ℌ is a subset whose linear span is dense in that Hilbert space. This means that the closure of the linear span of \(\ \{\phi_k(x)\}_{k \ge 0} \quad \) is the entire Hilbert space ℌ. Every element in ℌ can be approximated arbitrarily well by linear combinations of vectors from a complete system.
The term "complete" was introduced in 1910 by the Russian mathematician Vladimir Andreevich Steklov (1864--1926), a student of Alexander Lyapunov.
Example 7: There are known other orthogonal and complete sets of functions that are used in other than differential equations areas. In particular, Li--Torney system of step function is very useful in computer science. The collection of Walsh functions form a complete orthogonal set of functions that can be used to represent any discrete function---just like trigonometric functions can be used to represent any continuous function in Fourier analysis. These functions as well as the Walsh--Hadamard code are named after the American mathematician Joseph L. Walsh (1895--1973). Applications of the Walsh functions can be found wherever digit representations are used, including speech recognition, medical and biological image processing, and digital holography. The Haar wavelet is a sequence of rescaled "square-shaped" functions which together form a wavelet family or basis. Wavelet analysis is similar to Fourier analysis. The Haar sequence was proposed in 1909 by the Hungarian mathematician Alfréd Haar (1885--1933).
The Faber–Schauder system is a Schauder basis for the space ℭ([0, 1]) of continuous functions on [0, 1]. The Franklin system is obtained from the Faber--Schauder system by the Gram–Schmidt orthonormalization procedure.
■A subset S ⊂ ℌ is complete (or total) if its linear span is dense in ℌ. That is,\( \quad \overline{\mbox{span}(S)} = ℌ. \quad \) Alternatively, by the projection theorem, S is complete if and only if the only vector orthogonal to all vectors in is the zero vector (S⊥ = {0}).
Every dense set is a complete set. However, a complete set is generally NOT a dense set: A complete set only needs its linear combinations (finite sums, scaling, and their limits) to fill the space. The set of points themselves can be incredibly sparse, isolated, or discrete.
A Hilbert space is separable if and only if it contains a countable dense subset. Interestingly, this is perfectly mirrored by completeness: a Hilbert space is separable if and only if it possesses a countable complete orthonormal set (a Hilbert basis). While the dense subset itself must contain uncountably many points to cover every single neighborhood, the complete basis only needs countably many points because the vector addition and scalar multiplication do the heavy lifting of mapping out the rest of the space.
Example 8: The Weierstrass Approximation Theorem states that standard polynomials {1, x, x², x³, … , xⁿ, …} form a complete set in the Hilbert space 𝔏²([0, 1]). Therefore, every continuous real-valued function on a closed and bounded interval [𝑎, 𝑏] can be uniformly approximated as closely as desired by a polynomial functions.
If we create a sparse sequence of arbitrary real powers Λ = { λ₁, λ₂, λ₃, … }, when do the corresponding non-integer powers \( \displaystyle \quad \left\{ x^{\lambda_1} , x^{\lambda_2} , x^{\lambda_3} , \ldots \right\} \quad \) still manage to span the entire infinite-dimensional space? The answer depends entirely on the convergence of a simple numerical series.
Let us choose an strictly increasing sequence of positive real numbers to serve as arbitrary exponents: \[ \Lambda = \{ \lambda_1, \lambda_2, \lambda_3, \dots, \lambda_n, \dots \} \quad \text{where } 0 < \lambda_1 < \lambda_2 < \lambda_3 < \dots . \]
We now construct a highly sparse, custom power collection set \(S_\Lambda\): \[ S_\Lambda = \{ 1, x^{\lambda_1}, x^{\lambda_2}, x^{\lambda_3}, \dots, x^{\lambda_n}, \dots \} . \]
The set SΛ is complete in 𝕃²([0, 1]) if and only if the infinite sum of the reciprocal exponents diverges: \[ \sum_{n=1}^\infty \frac{1}{\lambda_n} = \infty . \]Two Striking Contrasts
- The Complete Prime Powers Set: Suppose we choose the exponents to be exclusively the prime numbers, \(\Lambda = \{2, 3, 5, 7, 11, \dots\}\). Because the sum of the reciprocals of the primes diverges (\(\sum \frac{1}{p} = \infty\)), the highly filtered set of sparse polynomials \( \{1, x^2, x^3, x^5, x^7, x^{11}, \dots\} \) is completely capable of approximating any square-integrable function perfectly in the mean square sense!
- The Incomplete Exponentially Sparse Set: Suppose we choose the exponents to grow exponentially fast, such as squaring the index: \(\Lambda = \{1, 4, 9, 16, 25, \dots, n^2, \dots\}\). Here, the reciprocal sum converges perfectly (\(\sum_{n=1}^\infty \frac{1}{n^2} = \frac{\pi^2}{6} < \infty\)). Because it converges, the resulting set \( \{1, x, x^4, x^9, x^{16}, \dots\} \) is not complete. The linear span of these curves leaves a massive infinite-dimensional "hole" in the Hilbert space that it can never reach.
Why this is pedagogically powerful:
Müntz's theorem shatters standard finite-dimensional geometric intuition. In ℝⁿ, removing a single basis vector drops the dimension immediately and changes the space permanently. In an infinite-dimensional space, we can drop an infinite number of coordinate dimensions (such as removing all composite powers like \(x^4, x^6, x^8, \dots\)), yet the remaining elements can still blend together perfectly through continuous linear combinations to span the entire original space!
To make the demonstration visually undeniable, the script attempts to approximate a simple target function—a smooth exponential curve, f(x) = ex—using two different polynomial basis sets up to a maximum index of N = 6:- The Complete Set (Slow-growing powers): Exponents grow linearly (λₙ = n), yielding standard polynomials {1, x, x², x³, x⁴, x⁵, x⁶}. The sum of reciprocals diverges \( \displaystyle \quad \left( \sum \frac{1}{n} = \infty \right) . \)
- The Incomplete Set (Fast-growing powers): Exponents grow quadratically (λₙ = n²), yielding sparse polynomials {1, x², x⁴, … , x36}. The sum of reciprocals converges: \( \displaystyle \quad \left( \sum \frac{1}{n^2} = \frac{\pi^2}{6} < \infty \right) . \)
In finite-dimensional linear algebra (ℝⁿ), a basis is simply a linearly independent set of vectors that spans the space via finite linear combinations. In an infinite-dimensional Hilbert space ℌ, however, the introduction of a metric and a topology changes this paradigm completely. We must carefully distinguish between two fundamentally different definitions of a "basis":
- Hamel Basis (Algebraic Basis): A subset B ⊂ ℌ such that every vector f ∈ ℌ can be written uniquely as a strictly finite linear combination of elements in B. While every vector space possesses a Hamel basis (by the Axiom of Choice), a Hamel basis for any infinite-dimensional Banach or Hilbert space is uncountably infinite and practically impossible to construct or utilize in physical applications.
- Hilbert Basis (Orthonormal / Topological Basis): An orthonormal set { e₁, e₂, e₃, …} such that every vector f ∈ ℌ can be represented uniquely as a limit of an infinite series converging in the norm topology: \[ f = \sum_{k=1}^\infty c_k \,\mathbf{e}_k \quad \text{where } c_k = \langle \mathbf{e}_k, f \rangle . \] Crucially, a Hilbert space is separable if and only if it possesses a countable Hilbert basis (such as the classical sines, cosines, or Zernike polynomials).
The concept of completeness is the exact mechanism that elevates a simple orthogonal collection of vectors into a legitimate topological Hilbert basis. The ultimate unity of these concepts is summarized by the following core theorem:
Theorem 2 (The Fundamental Criteria for a Hilbert Basis): Let { eₖ }, k ≥ 1, be an orthonormal sequence in a Hilbert space ℌ. The following statements are mathematically equivalent:
- The orthonormal set { eₖ }, k ≥ 1, is a complete system (i.e., if ⟨ f, eₖ ⟩ = 0 for all k ≥ 1, then f = 0.
- The linear span of the set is dense in the space (\(\overline{\text{span}(\{\mathbf{e}_k\})} = ℌ\)).
- The set forms a true Hilbert basis, allowing every element to be expanded as a mean-convergent Fourier series: \(f = \sum_{k=1}^\infty \langle e_k, f \rangle e_k\).
- Parseval's Identity holds perfectly for all elements in the space: \(\sum_{k=1}^\infty |\langle \mathbf{e}_k, f \rangle|^2 = \|f\|^2\).
To establish that these four criteria are fully equivalent, we will construct a circular chain of logical implications: \(\text{(1)} \implies \text{(2)} \implies \text{(3)} \implies \text{(4)} \implies \text{(1)}\).
Step 1: Prove \(\text{(1)} \implies \text{(2)}\) (Completeness implies Dense Linear Span)
Let \(M = \overline{\text{span}(\{e_k\})}\). We want to show that \(M = \mathcal{H}\). Suppose this is false, meaning the closed subspace \(M\) is a proper subset of \(\mathcal{H}\).
By the Projection Theorem in Hilbert spaces, any closed subspace allows the space to be decomposed as \(\mathcal{H} = M \oplus M^\perp\). If \(M \neq \mathcal{H}\), there must exist a non-zero vector \(g \in M^\perp\) such that \(g \neq 0\).
Step 2: Prove \(\text{(2)} \implies \text{(3)}\) (Dense Span implies Generalized Fourier Expansion)
Given an arbitrary vector \(f \in \mathcal{H}\), we construct its formal projection sequence \(S_n = \sum_{k=1}^n \langle e_k, f \rangle e_k\). From our prior derivation of Bessel's Inequality, we know the scalar series \(\sum_{k=1}^\infty |\langle e_k, f \rangle|^2\) converges and is bounded by \(\|f\|^2\). Because the coefficients are square-summable, the sequence of partial vectors \(\{S_n\}\) forms a Cauchy sequence in \(\mathcal{H}\), meaning it must converge to some vector \(f_0 \in \mathcal{H}\): \[ f_0 = \sum_{k=1}^\infty \langle e_k, f \rangle e_k \]
We must now prove that \(f_0 = f\). Let us evaluate the inner product of the difference vector \((f - f_0)\) against an arbitrary basis element \(e_j\): \[ \langle f - f_0, e_j \rangle = \langle f, e_j \rangle - \langle \sum_{k=1}^\infty \langle e_k, f \rangle e_k, e_j \rangle \] Due to the continuity of the inner product and the orthonormality property (\(\langle e_k, e_j \rangle = \delta_{k,j}\)), the right-hand summation collapses entirely to a single term: \[ \langle f - f_0, e_j \rangle = \langle f, e_j \rangle - \langle f, e_j \rangle = 0 \] Thus, \((f - f_0) \in \text{span}(\{e_k\})^\perp\). Statement (2) states that the linear span is dense, which means its orthogonal complement contains only the zero vector: \(\text{span}(\{e_k\})^\perp = \{0\}\). Therefore, \(f - f_0 = 0\), which yields the exact Fourier expansion \(f = \sum_{k=1}^\infty \langle e_k, f \rangle e_k\).
Step 3: Prove \(\text{(3)} \implies \text{(4)}\) (Fourier Expansion implies Parseval's Identity)
Assume every vector expands as \(f = \sum_{k=1}^\infty c_k e_k\), where \(c_k = \langle e_k, f \rangle\). We evaluate the squared norm by breaking it into its inner product definition: \[ \|f\|^2 = \langle f, f \rangle = \langle \sum_{k=1}^\infty c_k e_k, f \rangle \] Because the inner product is continuous, we can safely pull out the summation and the scalar coefficients. Remembering that the inner product is conjugate-linear in the first position (\(\langle \alpha x, y \rangle = \overline{\alpha}\langle x, y \rangle\)), this transforms into: \[ \|f\|^2 = \sum_{k=1}^\infty \overline{c_k} \langle e_k, f \rangle \] Substituting the definition of the geometric projection coefficient \(\langle e_k, f \rangle = c_k\) back into the expression reveals the scalar product of a complex number and its conjugate: \[ \|f\|^2 = \sum_{k=1}^\infty \overline{c_k} \cdot c_k = \sum_{k=1}^\infty |c_k|^2 = \sum_{k=1}^\infty |\langle e_k, f \rangle|^2 \] This successfully yields perfect Parseval equality for all vectors in the space.
Step 4: Prove \(\text{(4)} \implies \text{(1)}\) (Parseval's Identity closes back to Completeness)
We assume that Parseval's identity holds for all vectors in \(\mathcal{H}\). To test completeness, let us pick an arbitrary vector \(g \in \mathcal{H}\) and subject it to the zero-vector projection test, assuming that \(\langle e_k, g \rangle = 0\) for every single index \(k \ge 1\).
Applying Parseval's identity directly to this specific vector yields: \[ \|g\|^2 = \sum_{k=1}^\infty |\langle e_k, g \rangle|^2 = \sum_{k=1}^\infty |0|^2 = 0 \] Because the norm is strictly positive-definite (\(\|g\|^2 = 0 \iff g = 0\)), this forces the vector \(g\) to be the trivial zero vector. This satisfies the precise structural definition of completeness, successfully locking the circular proof chain. \(\blacksquare\)
Example 9: Let us consider the collection of cosine functions { 1, cos(x), cos(2x), cos(3x), … } on an interval of length 2π, which we know from the previous sections is orthogonal. We have
On interval [0, π], the cosine set {1, cosx), cos(2x), …} is fully complete (representing the Fourier Cosine Series). Similarly, the standalone sine set { sin(n x } is also fully complete on this exact same interval (representing the Fourier Sine Series, which uses Dirichlet boundary conditions ϕ(0) = ϕ(π) = 0.
Adding sin(x) to the complete set of cosine functions on [0, π] destroys its properties as a Hilbert basis. While it might seem intuitive that adding more vectors to a space-filling set would make it "more complete," it actually violates the core structural requirements of a basis. If you append sin(x) to the sequence {1, cosx), cos(2x), …} on the interval [0, π], the system undergoes two critical topological failures:
- Absolute Loss of Linear Independence (Overcompleteness) A Hilbert basis requires that every vector in the space can be expanded as a unique infinite series. By adding sin(x), you introduce redundancy. The function sin(x) is already a square-integrable function on [0, π]. Because the cosine set is already complete, sin(x) can already be perfectly expanded as an infinite series of cosines (specifically, its Fourier Cosine Series): \[ \sin (x) = \frac{2}{\pi} - \frac{4}{\pi}\,\sum_{n\ge 1} \,\frac{\cos (2nx)}{4 n^2 -1}\quad \mbox{for } x \in [0, \pi ]. \] Because sin(x) is a linear combination of the existing basis elements, adding it to the set makes the set linearly dependent. An arbitrary function would no longer have a unique expansion, which disqualifies the set from being a basis.
- Failure of Orthogonality A Hilbert basis relies on an orthonormal (or at least orthogonal) system so that you can extract coefficients easily via the 𝔏²-inner product: cₖ = ⟨f, eₖ⟩. The newly added function sin(x) is not orthogonal to the existing cosine functions. For example, calculating the inner product between sin(x) and the constant function 1 over [0, π] yields: \[ \langle 1 , \sin (x) \rangle = \int_0^{\pi} 1 \cdot \sin (x)\,{\text d}x = \left[ - \cos (x) \right]_{x=0}^{\pi} = 2 \ne 0. \] It also shares non-zero inner products with every even cosine term (cos(2n x)). This completely ruins the geometric decoupling that makes a Hilbert basis useful.
Frames are incredibly useful in signal processing and data transmission because their redundancy adds a layer of error resistance—if you lose a few coefficients, the remaining ones can still reconstruct the signal. However, in pure geometric terms, it ceases to be a Hilbert basis. ■
The Parseval Identity (Generalized)
Originally presented by the French mathematician Marc-Antoine Parseval (1755--1836) for series expansions, this identity is the ultimate culmination of geometric projection theory. It states the exact conditions under which **Bessel's Inequality transforms into a perfect metric equality**. Parseval's identity holds if and only if
In physical terms, Parseval's identity is the mathematical expression of the Conservation of Energy. It states that the total energy of a signal (computed as the continuous quadratic integral of its waveform in the spatial/time domain) is exactly equal to the aggregate sum of the energies of its individual frequency components (the discrete sum of its generalized Fourier-orthogonal coefficients). No energy is lost, provided the chosen basis is complete. It is also known as Rayleigh's energy theorem, or Rayleigh's Identity because the integral \( \displaystyle E[f] = \frac{1}{\ell} \int_{-\ell}^{\ell} \left\vert f (x) \right\vert^2 {\text d}x = \frac{\| f \|^2}{\ell}\ \) usually represents the energy in applications.
Theorem 3 (Parseval's Identity, 1799): Let ℌ be a Hilbert space, and let { e₁, e₂, … , eₙ, …} be a complete orthonormal basis for ℌ. Then for any vector f ∈ ℌ, the generalized Fourier coefficients cₖ = ⟨ eₖ, f ⟩ satisfy:
Let ℌ be a Hilbert space and let { e₁, e₂, … , eₙ, …} be a complete orthonormal basis. By definition of a Hilbert basis, the linear span of these vectors is dense in the space, meaning any arbitrary vector f ∈ ℌ can be expressed uniquely as an infinite linear combination of the basis vectors converging in the norm topology: \[ f = \sum_{k=1}^\infty c_k \,\mathbf{e}_k \quad \text{where}\quad c_k = \langle \mathbf{e}_k, f \rangle . \]
To evaluate the squared norm ∥f∥², we express it via the standard inner product definition \(\|f\|^2 = \langle f, f \rangle\). Substituting the convergent infinite series expansion into the left position of the inner product yields:
Because the inner product is a bounded, continuous linear operator relative to its scalar arguments, we can safely pull the summation and the scalar coefficients out of the bracket:
Recall that the generalized Fourier coefficients are defined geometrically as cₖ = ⟨ eₖ, f ⟩. However, since we defined our inner product to be conjugate-linear in the first argument, the scalar extraction follows the rule \(\langle \alpha x, y \rangle = \overline{\alpha}\langle x, y \rangle\). This gives us:
Substituting the projection definition ⟨ eₖ, f ⟩ = cₖ back into the expression reveals the product of a complex number and its complex conjugate:
Writing out the components explicitly in terms of the original geometric inner product projections finishes the proof:
The Role of Completeness: It is crucial to point out that if the sequence { eₖ } were incomplete, there would exist a non-zero vector g ∈ ℌ orthogonal to every element in the system (⟨ eₖ, g ⟩ = 0). For that vector, the left-hand summation would collapse entirely to \(0\) while the right-hand norm ∥g∥² > 0, failing the equality and reverting back to a strict Bessel inequality.
Corollary 3 (Classical Trigonometric Fourier Series): Let f(x) be a square-integrable function over the symmetric periodic domain [−ℓ, ℓ]. If its classical Fourier coefficients are given by \(a_n\) and \(b_n\), then Parseval's relationship locks into the absolute equation:
Technical Clarification: Rigor Behind the Summation Swap
In finite dimensions, pulling a summation out of an inner product is a trivial application of linearity: \(\langle \sum c_k e_k, f \rangle = \sum c_k \langle e_k, f \rangle\). However, in an infinite-dimensional Hilbert space, swapping an infinite summation with an inner product is not an algebraic law—it is a topological property driven by continuity.
Formally, an infinite series in a Hilbert space is defined as the limit of its partial sums: \(\sum_{k=1}^\infty c_k e_k = \lim_{n \to \infty} S_n\), where \(S_n = \sum_{k=1}^n c_k e_k\). The swap performed in the proof relies on the fact that the inner product is a continuous function in both of its arguments.
Why is Step A legal?
By the Cauchy Inequality, the inner product is bounded. Let \(v_n\) be a sequence of vectors converging to \(v\) in the norm topology (meaning \(\|v_n - v\| \to 0\)). The structural deviation of their inner products satisfies: \[ |\langle v_n, f \rangle - \langle v, f \rangle| = |\langle v_n - v, f \rangle| \le \|v_n - v\| \cdot \|f\| . \] As n → ∞, the term \(\|v_n - v\|\) vanishes, forcing \(|\langle v_n, f \rangle - \langle v, f \rangle| \to 0\). This proves that we can legally slide the limit operator completely outside the bracket: \(\langle \lim v_n, f \rangle = \lim \langle v_n, f \rangle\).
Why is Step B legal?
Once the limit is safely outside, the expression inside the inner product becomes a strictly finite sum \(S_n\). For finite sums, the algebraic properties of the inner product space hold unconditionally, allowing us to distribute the bracket across addition and pull out the conjugates of the scalar parameters.
⚠️ Classroom Warning: This topological bridge collapses in an incomplete inner product space (a pre-Hilbert space). Without completeness, the sequence of partial sums \(S_n\) can point toward a "hole" in the space, meaning \(\lim S_n\) may not exist as a valid vector, rendering the entire operation illegal.
Example 10: The Basel problem was first posed by the Italian mathematician Pietro Mengoli in 1644, it asked for the exact analytical value to which the infinite series of the reciprocals of the squares converge, \[ \sum_{n\ge 1} \m\frac{1}{n^2} = 1 + \frac{1}{4} + \frac{1}{9} + \frac{1}{16} + \cdots . \]
For nearly a century, this simple-looking series taunted Europe's absolute greatest mathematical minds. Titans of mathematics—including Gottfried Wilhelm Leibniz (the co-inventor of calculus) and the brilliant Bernoulli brothers (Jacob and Johann)—relentlessly attacked the problem but failed to find an exact value.
Jacob Bernoulli managed to prove that the series converges to a finite number less than 2, but lamented that finding its exact value was frustratingly difficult. Because the problem was aggressively popularized by the Bernoulli family, who lived and worked in Basel, Switzerland, it became known across the continent as the Basel Problem.
The monumental breakthrough came in 1734 when a 28-year-old Leonhard Euler from Saint Petersburg, Russia (who coincidentally was also born in Basel and had studied under Johann Bernoulli) announced the exact solution.
To provide a striking analytical example, let us look back at the periodic sawtooth wave function f(x) = x over the domain [−π, π] with a system scale ℓ = π.
As established in Example 5, the total continuous physical energy of this system is \(E[f] = \frac{2\pi^2}{3}\). Its Fourier cosine terms vanish (\(a_n = 0\)), and its sine coefficients are explicitly \(b_n = \frac{2(-1)^{n+1}}{n}\).
Because the classical trigonometric system is mathematically proven to be complete on this interval, Parseval's identity forces the infinite summation out of its previous inequality constraint and transforms it into a clean analytical bridge: \[ \sum_{n=1}^\infty \left( \frac{2(-1)^{n+1}}{n} \right)^2 = \frac{2\pi^2}{3} \quad \implies \quad \sum_{n=1}^\infty \frac{4}{n^2} = \frac{2\pi^2}{3} . \]
Dividing both sides by \(4\) yields Euler's legendary solution to the Basel Problem (the evaluation of the Riemann zeta function at \(s=2\)): \[ \sum_{n=1}^\infty \frac{1}{n^2} = \frac{\pi^2}{6} \approx 1.64493 . \]
You can run this script to observe the fundamental difference between Parseval and Bessel: instead of getting stuck below an impenetrable ceiling, the summation now converges perfectly to the continuous energy boundary, illustrating how completeness closes the topological metric gap.
Orthogonal Expansions
The ultimate goal of establishing a complete orthonormal basis { eₖ(x) }k ≥ 1 in an infinite-dimensional Hilbert space 𝔏²([𝑎, 𝑏], w) is to enable generalized Fourier expansions. Parseval's identity and Theorem 2 guarantee that any square-integrable function f(x) can be uniquely decomposed into an infinite series: \[ f(x) = \sum_{k=1}^\infty c_k \,\mathbf{e}_k (x) \quad \text{where } c_k = \frac{\langle f, \mathbf{e}_k \rangle}{\|\mathbf{e}_k\|^2} = \frac{\int_a^b \,f(x)\, \mathbf{e}_k(x) \,w(x)\,{\text d}x}{\int_a^b \, |\mathbf{e}_k(x)|^2 \,w(x)\,{\text d}x} . \]
Where do these complete, space-filling coordinate frameworks come from in physical applications? The answer is almost always the **Sturm–Liouville eigenvalue problem**. By adjusting the boundary conditions and domain constraints, we generate the entire family of classical orthogonal special functions used in physics and engineering.
The Continuum of Sturm–Liouville Differential Frameworks
The generalized Sturm–Liouville differential operator equation, \(\frac{\text{d}}{\text{d}x}[p(x)\phi'] + q(x)\phi + \lambda \,w(x)\,\phi = 0\), branches into three main structural archetypes based on the behavior of the coefficient maps at the boundary endpoints:
| Problem Category | Mathematical Conditions | Resulting Basis System | Physical Domain Type |
|---|---|---|---|
| Regular Classical | \(p(x) > 0, \; \rho(x) > 0\) across the entire closed interval \([a, b]\). Explicit separated bounds. | Classical Trigonometric Fourier Series (\(\sin, \cos\)) | Bounded, closed intervals (e.g., \([0, \ell]\)) |
| Singular (Bounded Endpoints) | \(p(x)\) vanishes smoothly to exactly \(0\) at one or both boundary endpoints (\(p(a)=0\) or \(p(b)=0\)). Bounds replaced by finiteness conditions. | Orthogonal Polynomials (Legendre, Chebyshev, Jacobi) | Naturally bounded geometries (e.g., Spherical/Disk horizons \([-1, 1]\)) |
| Singular (Unbounded Horizons) | The interval itself extends out to infinity (\(b \to \infty\)), or the weight metric density \(\rho(x)\) singularizes at the origin. | Bessel, Hermite, Laguerre Functions | Open radial fields, infinite quantum string spaces (\([0, \infty)\) or \((-\infty, \infty)\)) |
When solving partial differential equations over circular cylindrical systems (such as a vibrating drumhead or electromagnetic wave propagation through a fiber optic core), the radial coordinate equation takes the form of Bessel's singular differential equation: \[ x^2 \frac{\text{d}^2\phi}{\text{d}x^2} + x \frac{\text{d}\phi}{\text{d}x} + (\lambda x^2 - \nu^2)\phi(x) = 0 \implies \frac{\text{d}}{\text{d}x}\left[ x \frac{\text{d}\phi}{\text{d}x} \right] - \frac{\nu^2}{x}\phi + \lambda x \phi = 0 . \] Here, p(x) = x and the internal weight function is explicitly w(x) = x. Because p(0) = 0, the origin is a regular singular point. We drop the traditional unachievable separated boundary constraint at x = 0 and replace it with a regularity boundary clause: \(\lim_{x \to 0^+} |\phi(x)| < \infty\).
This condition immediately eliminates the second-kind Neumann functions (\(Y_\nu(x) \to -\infty\)), leaving exclusively the first-kind Bessel functions \(J_\nu(\alpha_{n} x)\). These scaled functions form a complete orthogonal basis under the radial area metric weight index x.
Example 11: To provide a clear, nontrivial computational challenge, let us reconstruct a discontinuous radial physical step-wave signal \(f(x)\) over a cylindrical aperture domain \(x \in [0, 1]\) using a zero-order (\(\nu=0\)) Fourier-Bessel series expansion: \[ f(x) = \begin{cases} 1.5 & \text{if } 0 \le x \le 0.5 \\ 0.2 & \text{if } 0.5 < x \le 1 \end{cases} \]
The expansion functions are chosen as \(e_n(x) = J_0(\alpha_n x)\), where \(\alpha_n\) is the \(n\)-th positive root satisfying the fixed boundary constraint \(J_0(\alpha_n) = 0\). Due to weight-metric orthogonality across the polar area metric space, the generalized projection coefficients evaluate to: \[ c_n = \frac{\int_0^1 f(x) J_0(\alpha_n x) x \,{\text d}x}{\int_0^1 [J_0(\alpha_n x)]^2 x \,{\text d}x} = \frac{2}{\left[J_1(\alpha_n)\right]^2} \int_0^1 f(x) J_0(\alpha_n x) x \,{\text d}x \]
It should point out that despite the target function dropping off sharply after x = 0.5, the approximation remains highly stable near the boundaries. This stability shows the geometric influence of the weight function w(x) = x, which down-weights errors near the origin while tightly controlling the structural fit near the perimeter boundary.
The Gibbs Phenomenon in Non-Trigonometric Spaces: Have students look closely at the oscillations surrounding the step transition at x = 0.5. This is an excellent way to show that the Gibbs Phenomenon is not a quirk unique to sine and cosine waves. It is a fundamental property of projecting discontinuous target states onto any truncated, infinite-dimensional continuous orthogonal basis.
■Example 11A: Expand the function
The solution of the differential equation \( y'' + \lambda\,y =0 \) may have one of three forms, depending on λ, so it is necessary to consider these cases. Let us start with λ = 0, the general solution becomes a linear function
If λ is negative, we can let λ = -μ² so that μ > 0. Then the differential equation for y becomes
Next consider λ > 0, then the general solution is
To find the expansion for f in terms of the eigenfunctions, we write
With the geometric framework of completeness and the Parseval identity established, we now possess the mathematical authorization to write down the generalized formula for reconstructing an arbitrary function \(f(x)\). Whether our chosen system yields trigonometric functions, Bessel functions, or classical orthogonal polynomials, the underlying algebraic engine remains identical.
If \(\{\phi_n(x)\}_{n=0}^{\infty}\) constitutes a complete orthogonal system in the weighted Hilbert space \(L^2([a,b], w(x)\text{d}x)\), any suitable function can be uniquely expanded using its generalized Fourier coefficients:
This elegant equation unifies several major classical expansions that appear across mathematical physics:
- Trigonometric Fourier Series: Arising from constant coefficients and periodic or Dirichlet boundary conditions.
- Bessel Series: Generated when solving problems in cylindrical coordinates where the weight function is \(w(x) = x\).
- Orthogonal Polynomials: Such as Legendre, Hermite, or Chebyshev systems, arising from unbounded intervals or singular boundary coordinates.
However, expressing an equality with an infinite sum of functions introduces a major topological hurdle. To safely manipulate these series in physical applications, we must answer a critical question: In what structural sense does this infinite sequence of graphs actually merge into our target function? To resolve this, we must examine the distinct Modes of Convergence.
Convergence of Orthogonal Expansions
Formula \eqref{EqOrtho.7} tells us that there is a mapping
When working with generalized Fourier series \(f(x) \sim \sum c_k \,\phi_k(x) , \ \) stating that a series "converges" is meaningless without specifying the topological metric. In Hilbert spaces, standard completeness guarantees convergence in the mean-square (\(\mathcal{L}^2\)) sense. However, physical applications often require stronger types of convergence, such as pointwise or uniform convergence. This final section outlines the precise conditions governing these transitions.
Our first fundamental result links operator theory directly to convergence strength. If a function is smooth enough to be actively processed by our differential operator, the convergence of its expansion locks into a highly robust state.
Theorem 4 (Domain Convergence for Self-Adjoint Operators): Let \(\mathcal{L}\) be an unbounded self-adjoint linear differential operator acting on a Hilbert space ℌ, and let { ϕₖ } (k ≥ 1) be its complete sequence of orthonormal eigenfunctions. If a function f belongs explicitly to the domain of the operator, \(f \in \mathcal{D}(\mathcal{L})\), then its generalized Fourier series converges to \(f\) in the graph norm. Consequently, the residual error vanishes rapidly, guaranteeing absolute and uniform convergence if the operator is elliptic.
For any arbitrary element f ∈ ℌ, its generalized Fourier coefficients are defined as cₖ = ⟨ f , ϕₖ⟩.
Assume that f belongs explicitly to the domain of the operator, \(f \in \mathcal{D}(\mathcal{L})\). This means that the image vector g = ℒf is a well-defined element within the Hilbert space ℌ (g ∈ ℌ). We can compute the generalized Fourier coefficients of g, denoted by dₖ, via the standard inner product: \[ d_k = \langle g , \phi_k \rangle = \left\langle \mathcal{L}f , \phi_k \right\rangle . \] Because ℒ is a self-adjoint operator, we are permitted by definition to shift the operator action across the inner product bracket to act upon the eigenfunction: \[ d_k = \left\langle f , \mathcal{L} \phi_k \right\rangle . \] Substituting the eigenvalue relation ℒϕₖ = λₖϕₖ yields \[ d_k = \left\langle f , \lambda_k \phi_k \right\rangle = \lambda_k\,\left\langle f , \phi_k \right\rangle =\lambda_k \,c_k . \] Since g = = ℒf is a guaranteed vector in the Hilbert space ℌ, we can apply Bessel's Inequality directly to its Fourier coefficient string { dₖ} \[ \sum_{k\ge 1}\,\left\vert d_k \right\vert^2 = \sum_{k\ge 1}\,\left\vert \lambda_k\,c_k \right\vert^2 = \sum_{k\ge 1}\,\lambda_k^2 \left\vert c_k \right\vert^2 \le \| \mathcal{L}f \|^2 < \infty . \] This result states that the sequence ( λₖcₖ } is square-summable. Because |λₖ| → ∞ as k → ∞, the coefficients cₖ must decay at an accelerated rate that outpaces the growth of the eigenvalues.
To prove convergence in the graph norm (and hence the domain), we analyze the partial sums \( \displaystyle \quad S_n (x) = \sum_{k=1}^n c_k \,\phi_k (x) . \quad \) The remainder error in the operator energy norm is given by: \[ \left\| \mathcal{L}f - \mathcal{L} S_n \right\|^2 = \left\| \mathcal{L}f - \sum_{k=1}^n \,c_k\,\lambda_k \,\phi_k \right\|^2 = \sum_{k\ge n+1} \ \lambda_k^2 \,|c_k |^2 . \] Because \( \displaystyle \quad \sum_{k\ge 1}\,\lambda_k^2 \,|c_k |^2 \quad \) is a convergent series of real numbers, its tail must vanish as n → ∞. \[ \lim_{n\to\infty}\ \left\| \mathcal{L}f - \mathcal{L} S_n \right\|^2 = \lim_{n\to\infty}\ \sum_{k\ge n+1}\ \lambda_k^2 \,| c_k |^2 = 0 . \] Thus, the generalized Fourier series converges to f not only in the standard baseline 𝔏²-norm, but also under the strict action of the operator. If ℒ is a second-order elliptic differential operator, standard Sobolev embedding theorems dictate that this energy convergence immediately implies that the expansion series converges absolutely and uniformly across the continuous spatial domain.
Classroom Note: The Implicit Compact Resolvent
An Analytical Subtlety: Advanced students might notice that the mathematical proof relies heavily on the fact that the eigenvalues grow to infinity (|λₖ| → ∞), which is a direct consequence of the operator having a compact resolvent. Why then is "compact resolvent" not explicitly written in the statement of Theorem 4?
The condition is omitted because it is already mathematically implied by the phrase "complete sequence of orthonormal eigenfunctions."
For an unbounded self-adjoint operator \(\mathcal{L}\), the spectrum can split into a discrete spectrum (isolated points) and a continuous spectrum (solid bands). An operator possesses a compact resolvent if and only if its spectrum is purely discrete, forcing the eigenvalues to diverge to infinity without accumulating at any finite point. By explicitly assuming that \(\mathcal{L}\) possesses a countable, discrete sequence of eigenfunctions that completely spans the Hilbert space ℌ, we have already implicitly forced the operator's resolvent to be compact.
Remarks on the graph norm: The term graph norm has two primary meanings depending on whether you are working in functional analysis (mathematics) or deep learning (graph neural networks). In functional analysis, the graph norm is a special norm defined on the domain of a linear operator (often an unbounded operator.
For a linear operator T mapping from a Banach space X to Y with domain 𝐷(T), the graph norm of a vector x ∈ 𝐷(T) is given by
The credit for proving that a function belonging to the domain of a self-adjoint differential operator has an absolutely and uniformly convergent eigenfunction expansion belongs to the legendary Italian mathematician Mauro Picone (1885–1977) and, in its most complete classical form, to a prominent Russian and Soviet applied mathematician, physicist, and mechanician Vladimir Steklov (1864–1926).
Long before abstract Hilbert spaces were formalized, Vladimir Steklov aggressively studied what he called the "closure problem" for orthogonal polynomials and sines/cosines. He proved that if a function has a continuous second derivative and satisfies the boundary conditions (meaning it sits in what we now call ), its Fourier series converges uniformly.
Mauro Picone established foundational inequalities regarding self-adjoint differential equations. He rigorously showed that applying the differential operator directly to a sufficiently smooth function generates square-summable boundaries that force accelerated coefficient decay.
The Hungarian and American mathematician, physicist, computer scientist and engineer John von Neumann (1903–1957) was the first to formalise the concept of an unbounded self-adjoint operator acting on an infinite-dimensional Hilbert space to provide a bulletproof mathematical framework for quantum mechanics. In 1929–1930, von Neumann lifted the concrete calculus works of Steklov, Picone, David Hilbert, and Erhard Schmidt into the abstract mathematical stratosphere by defining the domain as a closed linear manifold in an abstract Hilbert space, demonstrating that the "smoothness" required by Steklov was actually an expression of the graph norm topology.
Perspective Note: The 1D Baseline and the Curricular Horizon
A Word on Scope for ODE Students: The abstract machinery of Hilbert spaces, operator domains \(\mathcal{D}(\mathcal{L})\), and ellipticity can initially feel incredibly dense. Because this is a course on Ordinary Differential Equations (ODEs), we will focus our attention strictly on the one-dimensional (1D) case.
In 1D, the spatial operator is simply a standard second-order derivative mapping (such as \(\mathcal{L} = -\frac{\text{d}^2}{\text{d}x^2} \ \)), and the geometric domains are straightforward, closed intervals like \([0, \pi]\) or \([-1, 1]\). Rest assured that mastering this 1D framework is the crucial first step; the algebraic steps, inner product projections, and coefficient decay principles you learn here are perfectly preserved.
Looking Ahead to PDEs: When you advance to future courses in Partial Differential Equations (PDEs) and advanced mathematical physics, you will see this exact same structural blueprint expand into multi-dimensional spaces (ℝ² and ℝ³). The ordinary derivatives will evolve into gradient, divergence, and Laplacian field operators acting over intricate geometric shapes (such as vibrating circular plates or fluid horizons).
Whether analyzing the 1D classical string models in this class or calculating high-order biharmonic elastic deformations (Δ²u) in graduate school, the underlying Hilbert space properties remain identical. By solidifying your grasp of 1D expansions today, you are building the core framework necessary to navigate higher-dimensional physical fields tomorrow.
Pedagogical Insight: To be in \(\mathcal{D}(\mathcal{L})\), the function \(f\) must not only be square-integrable, but it must also be differentiable up to the order of the operator, and it must satisfy the system's boundary conditions perfectly. When these structural conditions are met, the coefficients \(c_k\) decay exceptionally fast because \(\langle \mathcal{L}f, \phi_k \rangle = \lambda_k \langle f, \phi_k \rangle\), ensuring smooth, powerful convergence.
Bibliography & Comprehensive References
The following authoritative texts and historical foundations govern the spectral architecture, domain convergence metrics, and completeness frameworks developed throughout this instructional module.
I. Functional Analysis & The Spectral Theorem
-
Reed, M., & Simon, B. (1980).
Methods of Modern Mathematical Physics: Vol 1: Functional Analysis (Revised ed.). Academic Press.
Core Mapping: Chapter VIII (Bounded and Unbounded Operators / The Spectral Theorem). This is the gold standard for defining the rigorous domain frameworks \(\mathcal{D}(\mathcal{L})\) of unbounded self-adjoint operators. -
Conway, J. B. (2019).
A Course in Functional Analysis (2nd ed.). Springer Graduate Texts in Mathematics.
Core Mapping: Chapter X (Unbounded Operators). Provides the definitive abstract proofs linking the compact resolvent property directly to purely discrete spectra and countable eigenfunction sequences.
II. Differential Equations & Orthogonal Expansions
-
Stakgold, I., & Holst, M. (2011).
Green's Functions and Boundary Value Problems (3rd ed.). John Wiley & Sons.
Core Mapping: Chapters 4 & 7 (Inner Product Spaces, Generalized Fourier Series, and Sturm–Liouville Problems). Contains the highly explicit, student-accessible proof of Theorem 4, illustrating how \(\sum \lambda_k^2 |c_k|^2 < \infty\) drives absolute and uniform convergence.
III. Historical Foundations
-
Steklov, V. A. (1904).
"Sur les problèmes fondamentaux de la physique mathématique," Annales Scientifiques de l'ÉCOLE Normale Supérieure.
Historical origin of the "closure problem", proving uniform eigenfunction convergence via explicit classical calculus boundaries before the invention of modern abstract spaces. -
Picone, M. (1911).
"Sulle equazioni differenziali lineari ordinarie del secondo ordine," Rendiconti del Circolo Matematico di Palermo.
Establishes early versions of the integral inequalities that dictate how structural smoothness bounds control generalized expansion limits. -
von Neumann, J. (1930).
"Allgemeine Eigenwerttheorie Hermitescher Funktionaloperatoren," Mathematische Annalen.
The monumental work that geometricized classical analysis by formally mapping operator domains into abstract linear manifolds within Hilbert spaces.
Example 12: Let us test the regular self-adjoint operator \(\mathcal{L} = -\frac{\text{d}^2}{\text{d}x^2} \quad \) on the interval [0, π] subject to strict Dirichlet boundary conditions: ϕ(0) = ϕ(π) = 0. The complete orthonormal eigenfunction set is given by \(\phi_k(x) = \sqrt{\frac{2}{\pi}}\sin(kx)\ \) with eigenvalues λₖ = k².
We choose a target function that matches these conditions perfectly: \(f(x) = x^2(\pi - x)\). Since \(f(0)=0\), \(f(\pi)=0\), and its second derivative is continuous, \(f \in \mathcal{D}(\mathcal{L})\). Theorem 4 guarantees that the Fourier coefficients will decay exceptionally fast, eliminating edge oscillations.
Example 13: We keep the exact same operator and basis as Example 12, but change the target function to a smooth parabola that violates the boundary conditions: f(x) = x(π − x) + 2. Note that f(0) = 2 ≠ 0 and f(π) = 2 ≠ 0.
Even though this function is smooth and infinitely differentiable, it does not belong to \(\mathcal{D}(\mathcal{L})\) because it breaks the domain's boundary rules. As a result, the convergence guarantees of Theorem 4 collapse, leading to slow coefficient decay and persistent edge errors.
Theorem 5 (The Spectral Theorem): Let Let \(\mathcal{L}\) be the formal differential expression:
If \( \mathcal{L} \) is realized as a self-adjoint operator on its maximal domain \( \mathcal{D}(\mathcal{L}) \subset \) 𝔏²([𝑎, 𝑏], ρ), then its spectrum consists purely of a countable set of real eigenvalues λₙ → ∞ accumulating only at infinity, and the corresponding eigenfunctions form a complete orthonormal basis for the weighted space 𝔏²([𝑎, 𝑏], ρ).
To prove that the spectrum is purely discrete (no continuous spectrum walls), we must show that the resolvent operator \( R_{\gamma} = \left( \mathcal{L} + \gamma\,I \right)^{-1} \ \) is a compact operator from 𝔏²([𝑎, 𝑏], ρ) to 𝔏²([𝑎, 𝑏], ρ). By definition, the resolvent is a bounded map from 𝔏²([𝑎, 𝑏], ρ) into the Sobolev space \( H^1_{p, \rho}([a, b]) .\ \) Therefore, if the embedding of \( H^1_{p, \rho}([a, b]) \ \) into 𝔏²([𝑎, 𝑏], ρ) is compact, the resolvent Rγ will be compact.
Step 1: Bounding via the Weighted Resolvent. Assume a real value γ is chosen such that the shifted operator \( \mathcal{L} + \gamma\,I \ \) is strictly positive. For any f ∈ 𝔏²([𝑎, 𝑏], ρ), we examine the resolvent equation:
Step 2: Compactness via Weighted Sobolev Embedding. To prove Rγ is compact without relying on a perfectly continuous Green's function, we look at the energy norm of the domain:
The Rellich–Kondrachov Theorem for Weighted Spaces generalizes the classical compact embedding of Sobolev spaces into Lebesgue spaces by incorporating weight functions w(x) into the norms. Unlike the unweighted setting, a compact embedding in a weighted space does not automatically hold on bounded domains; it depends entirely on the behavior of the weights (such as whether they degenerate or blow up near the boundary) and the geometry of the domain.
The classical unweighted Rellich–Kondrachov theorem originates from a lemma by Franz Rellich (1930) and was proved for general Sobolev spaces by Vladimir Kondrashov (1945).
By the Rellich–Kondrachov Theorem for Weighted Spaces, if the boundary singularity behaves mild enough such that \( \displaystyle \quad \int_a^b \,\frac{{\text d}x}{\rho (x)} \quad \) or a similar structural weight integral is locally integrable near the boundary (for Bessel,\int_0^1 \,\frac{{\text d}x}{x} \quad \) diverges, but the corresponding solution multiplier \( x \cdot |y' | \ \) remains controlled because y′(0) = 0), the embedding of \( H^1_{p, \rho} \) into 𝔏²([𝑎, 𝑏], ρ) is compact.
Since the resolvent operator Rγ is a bounded mapping from 𝔏²([𝑎, 𝑏], ρ) into \( H^1_{p, \rho} \), composing it with this compact embedding proves that is a compact, self-adjoint operator on 𝔏²([𝑎, 𝑏], ρ).
Step 3: Spectral Mapping. Applying the classical Spectral Theorem for Bounded Compact Self-Adjoint Operators directly to Rγ yields a complete orthonormal basis of eigenfunctions ϕₙ(x) with eigenvalues μₙ → 0. Mapping these back via λₙ = 1/μₙ − γ yields the countable, discrete pure point spectrum λ#8345; → ∞ for the singular system.
Example 14: To illustrate the pure point spectrum and show how the eigenfunctions form an expanding complete basis, we can use a non-trivial Sturm–Liouville system with a variable potential coefficient:
- Domain: (0, ∞).
- Boundary Singularity: , which vanishes at the left boundary ( ). This marks the problem as singular.
- Nontrivial Weight Function: ρ(x) = e−x.
The following script numerically verifies this singular weighted problem. It computes the eigenvalues using a numerical differential solver, confirms that the inner products vanish when evaluated against the weight function e−x, and plots the resulting basis shapes.
Even though the interval is infinite, the space behaves like a compact one because the potential energy term effectively grows at infinity. If we change variables to eliminate the weight function via u(x) = y(x) e−x/2, the operator transforms into a Schrödinger-type operator,
Notice that as x → ∞, the potential V(x) → ∞. By the Rellich–Kondrachov Theorem for Unbounded Domains, a Schrödinger operator whose potential tends to infinity at infinity forces the trajectories of bounded energy functions to be tightly confined. They cannot leak out to infinity. This confinement guarantees that the injection map \{ H^1_{p, \rho}([a, b]) \to \) 𝔏²([𝑎, 𝑏]) is strictly compact.
- Because \{\ \lim_{x\to\infty}\ V(x) = \infty , \ \) the embedding is compact.
- Therefore, the resolvent \( \ R_{\gamma} = \left( \mathcal{L} + \gamma\,I \right)^{-1} \ \) is a compact, self-adjoint operator on 𝔏²([𝑎, 𝑏]).
- By the standard Hilbert–Schmidt theorem for compact operators (which does not assume anything about singular differential systems), Rγ has a countable set of discrete eigenvalues accumulating only at zero, and its eigenfunctions form a complete orthonormal basis.
- Inverting this maps the system directly back to \( \mathcal{L} \), proving that the singular Laguerre system has a pure point spectrum (λₙ = n) and its polynomials (Lₙ(x) form a complete basis fo r 𝔏²((0, ∞), e−x).
Theorem 6 (Generalized Riemann–Lebesgue Lemma): If \(\{\phi_n(x)\}_{n=0}^{\infty}\quad \) is a complete orthonormal system generated by a self-adjoint operator on \([a, b]\), then for any square-integrable function f ∈ 𝔏²([𝑎, 𝑏]), the generalized projections must vanish asymptotically:
Let ℌ = 𝔏²([𝑎, 𝑏], ρ) be a weighted Hilbert space over a domain [𝑎, 𝑏] (which can be regular or singular/unbounded) with a strictly positive weight function ρ(x) > 0 almost everywhere. Let { ϕₙ(x) }, n = 0, 1, 2, …, be a complete orthonormal system in ℌ, meaning:
Step 1: Geometric Orthogonal Projection in Finite Dimensions. Consider the finite partial sum vector representing the orthogonal projection of f onto the span of the first n basis functions:
Step 2: Deriving Bessel's Inequality. Consider the distance (error vector) between the true function f and its finite partial projection Sₙ. Because a norm squared is strictly non-negative, we can write:
Step 3: Taking the Infinite Limit and Applying the Divergence Test. Crucially, Bessel's inequality holds true for any finite integer n. The left-hand side, ∥f∥, is completely independent of n and represents the finite global energy of the function f ∈ 𝔏²([𝑎, 𝑏], ρ).
Taking the limit as n → ∞, the sequence of partial sums forms a monotonically increasing sequence bounded from above by the finite energy of the function:
Example 15: Let the domain be the interval I = [0, 1], and select the non-trivial weight function: ρ(x) = x. Consider a simple, constant, square-wave initial condition representing a uniform physical load or signal step over the domain:
The orthonormal basis functions for this specific weighted space are given by normalized Bessel functions of the first kind (order zero):
Now we evaluate how these modal numbers behave as the frequency tracking index goes to infinity. From the mathematical physics definition of Bessel zeros, the roots rₙ expand linearly as n scales up:
The following code performs three tasks to visually verify the Generalized Riemann–Lebesgue Lemma:
- It calculates the exact analytical weighted projection coefficients (cₙ) using the roots of the Bessel function.
- It numerically verifies that the sum of the squared coefficients satisfies Bessel's Inequality (and converges toward the total energy of the signalp).
- It generates a discrete stem plot that clearly visualizes how the amplitudes asymptotically vanish toward zero as the modal index scales up to 30.
Leveraging our complete basis, we can project any arbitrary function inside our space to build an approximation that minimizes total energetic variance.
Theorem 7 (Mean-Square Convergence): If \(\left\{ \phi_n (x) \right\}_{n\ge 0} \quad\) is a complete orthogonal system of functions on the interval [𝑎, 𝑏], then every square-integrable function f ∈ 𝔏²([𝑎, 𝑏]) has the canonical expansion:
The infinite expansion sum converges strictly in the 𝕃² mean-square sense, meaning the global integrated error norm vanishes completely:
Let ℌ = 𝔏²([𝑎, 𝑏], ρ) be a weighted Hilbert space over an interval [𝑎, 𝑏], equipped with the inner product ⟨· , ·⟩ and the associated norm ∥ · ∥. Let { ϕₖ } be a complete orthogonal system of functions in ℌ. For any arbitrary function f ∈ ℌ, we define its generalized Fourier coefficients by projecting f onto the basis elements:
Let \( \displaystyle \quad T_n = \sum_{k=0}^n \ \alpha_k\,\phi_k \quad \) be an arbitrary linear combination with arbitrary coefficients αₖ. We expand the distance metric ∥f − Tₙ∥²:
Step 2: Closure and Cauchy Sequence Verification. From the minimization identity above, since a norm squared cannot be negative, we obtain the weighted version of Bessel's Inequality for any finite n:
Step 3: Invoking the Completeness Axiom to Prove g = f. We must now verify that the limit function g is identically equal to our target function f. Let us evaluate the inner product of the error function (g = f) against any arbitrary basis element ϕₗ:
Example 16: Let our target function be the signum (step) function:
The standard Legendre polynomials Pₙ(x)
form a complete orthogonal system on [−1, 1]
with the normalization constant ∥Pₙ(x)∥ = 2/(2n + 1). The generalized projection coefficients are calculated via:
The following code constructs the Legendre expansion of the step function, calculates the exact global energy error for increasing values of n, and plots the results.
Important remark: The orthogonal expansion \( \displaystyle f(x) \sim \sum_{n\ge 0} c_n f_n (x) \) for a complete orthogonal system on [𝑎, b], holds in 𝔏² sense, but not necessarily pointwise, i.e. for a fixed x∈[𝑎, b] the series on the right hand side might not necessarily converge and, even if it does, it might not converge to f(x).
By restricting the regular domain parameters of our target signal, we lock down exactly how the series handles specific coordinate assignments and boundary step transitions. What happens when a function does not belong to \(\mathcal{D}(\mathcal{L})\)? For example, what if it contains step discontinuities or sharp corners? Here, we must turn to classical harmonic analysis.
Theorem 8 (Pointwise Sturm–Liouville Convergence): Let \(\{\phi_n(x)\}_{n=0}^{\infty} \ \) represent the explicit eigenfunctions of a regular, self-adjoint Sturm–Liouville operator defined on the compact interval.
If the target function \(f(x)\) and its derivative \(f'(x)\) are piecewise continuous across the closed boundary domain [𝑎, 𝑏], then the generalized series expansion:
converges pointwise to the exact local geometric arithmetic mean of directional limits at each coordinate within the open interval (𝑎, 𝑏):
Step A: Construct the Integral Kernel. We substitute the integral formula for the coefficients \( \displaystyle \quad c_k = \int_a^b \,f(x)\,\phi_k (x)\,w(x)\,{\text d}x \quad \) back into the finite sum expression:
Step B: Apply the Christoffel–Darboux Identity. Unlike sines/cosines where the kernel collapses via trigonometric sum laws, orthogonal polynomials collapse using a special 3-term recurrence identity discovered by Elwin Bruno Christoffel and Jean-Gaston Darboux:
. Step C: The Generalized Riemann–Lebesgue Localization Lemma. By splitting the integral around a tiny neighborhood of the point of interest x, the term 1/(x − t) acts exactly like the denominator in standard Fourier proofs.
If function f satisfies Dirichlet's conditions, there are finite number of subintervals where this function is monotonic. Then we can apply the Second Mean Value Theorem for Integrals, trapping the localized oscillations of the high-order functions ϕₙ(t) and forcing the high-frequency integral tails to drop cleanly to zero as n → ∞. This isolates the value exactly at the jump midpoint {f(x + 0) + f(x − 0)}/2.
Inside the Proof: The Christoffel–Darboux Kernel & Stieltjes Integration
For classical sines and cosines, proving pointwise convergence at a jump discontinuity relies on collapsing the series into the trigonometric Dirichlet Kernel. For general classical orthogonal polynomials, we no longer have trigonometric angle addition laws. Instead, the proof of Theorem 5 hinges on a remarkable algebraic identity known as the Christoffel–Darboux Formula.
When we substitute the projection coefficients back into a truncated expansion \(S_n(x) = \sum_{k=1}^n \,c_k \,\phi_k(x) ,\ \) the finite sum transforms into an integral equation wrapped around a localized density kernel: \[ S_n(x) = \int_a^b f(t) \left[ \sum_{k=1}^n \phi_k(x)\,\phi_k(t) \right] \,w(t)\,{\text d}t = \int_a^b \,f(t) \,K_n(x,t) \,w(t)\,{\text d}t . \] The Christoffel–Darboux Identity collapses this highly complicated polynomial summation into a single, elegant fractional expression: \[ K_n(x,t) = \sum_{k=1}^n \,\phi_k(x)\,\phi_k(t) = \frac{k_n}{k_{n+1}} \cdot \frac{\phi_{n+1}(x)\,\phi_n (t) - \phi_n (x)\,\phi_{n+1}(t)}{x - t} , \] where kₙ represents the leading coefficient of the \(n\)-th normalized polynomial degree.
The Analytical Mechanism: Riemann–Stieltjes Integration
Because the kernel \(K_n(x,t)\) oscillates rapidly, a rigorous convergence proof for a function \(f(t)\) of bounded variation cannot simply rely on splitting the function into monotonic components via algebraic addition. Doing so fails because the kernel does not have a uniform sign.
Instead, the proof treats the system within the framework of Riemann–Stieltjes integration. By framing the expansion integral against the function profile, analysts apply a specialized form of integration by parts where the variation is bound directly by the total variation measure \({\text d}f(t)\). This allows the singular denominator term \(\frac{1}{x-t}\) to be controlled. As \(n \to \infty\), the high-frequency polynomial oscillations outside the immediate neighborhood of \(x\) integrate to zero, forcing the series to settle perfectly onto the local arithmetic jump mean: \(\frac{1}{2}[f(x^+) + f(x^-)]\).
Explicit Parameters for Classical Systems
The table below provides the explicit values for the leading coefficients \(k_n\) and normalized basis vectors \(\phi_n(x)\) that drive the Christoffel–Darboux engine:
| System Family | Normalized Basis Element \(\phi_n(x)\) | Leading Coefficient \(k_n\) Value |
|---|---|---|
| Legendre Domain: \([-1,1]\), \(\rho=1\) |
\(\sqrt{\frac{2n+1}{2}} P_n(x)\) | \(\sqrt{\frac{2n+1}{2}} \frac{(2n)!}{2^n (n!)^2}\) |
| Chebyshev (1st Kind) Domain: \([-1,1]\), \(\rho=\frac{1}{\sqrt{1-x^2}}\) |
\(\sqrt{\frac{2}{\pi}} T_n(x) \quad (n \ge 1)\) | \(\sqrt{\frac{2}{\pi}} 2^{n-1} \quad (n \ge 1)\) |
| Jacobi Domain: \([-1,1]\), \(\rho=(1-x)^\alpha(1+x)^\beta\) |
\(\frac{1}{h_n^{1/2}} P_n^{(\alpha,\beta)}(x)\) where \(h_n\) is the standard normalization factor |
\(\frac{1}{h_n^{1/2}} \frac{1}{2^n} \frac{\Gamma(2n+\alpha+\beta+1)}{n!\Gamma(n+\alpha+\beta+1)}\) |
*Note: Standard Trigonometric Fourier bases collapse via trigonometric identity laws rather than algebraic polynomial recurrences, meaning they do not possess a formal $k_n$ algebraic parameter. Associated Legendre systems are typically mapped directly onto Jacobi systems with parameters \(\alpha = \beta = m\) to preserve algebraic polynomial properties.
IV. Specialized References & Foundations for Theorem 5
The following historical and graduate-level mathematical literature covers the definitive proofs regarding the Christoffel–Darboux identity, Dirichlet-Jordan structures, and the pointwise mean-value bounds of non-trigonometric expansions.
-
Szegő, G. (1975).
Orthogonal Polynomials, 4th edition/reprint (American Mathematical Society Colloquium Publications, Vol. 23). American Mathematical Society.
Core Mapping: Chapter III (The Christoffel–Darboux Formula) and Chapter IX (Convergence of Fourier Series of Orthogonal Polynomials). This masterwork is the definitive global reference for proving pointwise limits of Legendre, Jacobi, and Chebyshev polynomial strings under bounded variation constraints. -
Freud, G. (1971).
Orthogonal Polynomials (1st ed.). Pergamon Press.
Core Mapping: Chapter IV (Convergence Theory of Orthogonal Polynomial Series). Provides highly detailed, intermediate-level proofs showing how Jordan's localized bounded variation constraints function safely inside bounded intervals. -
Zygmund, A. (2002).
Trigonometric Series: Volumes I & II Combined (3rd ed.). Cambridge University Press.
Core Mapping: Volume 1, Chapters II & III. Contains the comprehensive classical analysis proofs for the Dirichlet–Jordan localization lemma, laying the exact foundational framework utilized to handle jump discontinuities across generic Hilbert spaces.
Example 17: We consider a classical step function \[ H(x) = \begin{cases} 1, &\quad \mbox{for } x > 0 , \\ 1/2 , &\quad \mbox{if }x = 0, \\ 0, &\quad \mbox{when } x < 0. \end{cases} \] We use Legendre Polynomials on [−1, 1], and employ Mathematica. . Theorem 5 guarantees that the polynomial string will cross exactly through the arithmetic mean (y = ½) at the jump coordinate.
The most remarkable result regarding pointwise convergence almost everywhere was obtained by Hans Rademacher (1922) and Dmitrii Menshov (1923).
Throughout this section, we have observed a recurring theme: standard square-integrability (f ∈ 𝔏², which means \(\sum |c_n|^2 < \infty\)) guarantees convergence in the mean-square sense, but it does not guarantee pointwise convergence. To obtain uniform pointwise convergence in Theorem 4, we had to restrict ourselves to smooth functions inside the operator domain \(\mathcal{D}(\mathcal{L})\).
The **Rademacher–Menshov Theorem** (1922–1923) answers a much more fundamental question in real analysis: What is the minimum coefficient decay rate required to ensure that a generalized Fourier series converges pointwise almost everywhere (a.e.) without assuming anything about the continuity or smoothness of the function?
The Logarithmic Brake: Rademacher and Menshov discovered that by multiplying the energy coefficients by a specific logarithmic weight factor, \(\log^2(1+n)\), the high-frequency variations are dampened just enough to ensure pointwise sanity almost everywhere.
A Historically Sharp Boundary: Why is this factor so celebrated? Dmitrii Menshov later proved that this theorem is mathematically sharp. For a completely arbitrary orthogonal system, you cannot replace \(\log^2(1+n)\) with any slower-growing sequence; if you drop the exponent even slightly, you can construct malicious orthogonal systems where the series diverges violently on a set of positive measure. This stands as a monument in descriptive set theory and the harmonic analysis of arbitrary orthogonal frameworks.
V. Primary Historical & Modern References for the Rademacher–Menshov Theorem
The following original articles and modern monographs document the discovery, sharpness proofs, and integration tracking metrics governing almost everywhere convergence boundaries.
-
Rademacher, H. (1922).
"Einige Sätze über Reihen von allgemeinen Orthogonalfunktionen," Mathematische Annalen, 87(1), 112–138.
The original breakthrough publication introducing the logarithmic weight bound to control the maximum fluctuations of general orthogonal expansion tail sums. -
Menchoff, D. (1923).
"Sur les séries de fonctions orthogonales," Fundamenta Mathematicae, 4(1), 82–105.
Independently establishes the formal limit properties and lays down the pioneering structural proofs showing the exact sharpness limits of the \(\log^2(n)\) criteria. -
Kashin, B. S., & Saakyan, A. A. (1989).
Orthogonal Series (Translations of Mathematical Monographs). American Mathematical Society.
Modern Monograph Standard: Chapter 3 (Convergence of Orthogonal Series Almost Everywhere). An exceptional graduate-level reference that features a meticulously complete, modern step-by-step breakdown of the Rademacher–Menshov localization lemma and its modern extensions.
Step 1: The Dyadic Splitting Strategy. Instead of tracking every single index step by step, we isolate the partial sums along a dyadic sequence of indices. Let k be a positive integer, and consider indices trapped within the dyadic window 2k < m ≤ 2k+1.
Any index m in this range can be written uniquely in binary format relative to its lower dyadic floor:
Step 2: Bounding via Cauchy–Bunyakovsky inequality. Applying the Cauchy–Bunyakovsky inequality (also is known as the Cauchy–Bunyakovsky–Schwarz inequality) to this binary expansion introduces a strategic weight factor j. For any x ∈ I:
Step 3: Integrating over the Domain Space. We integrate this maximum discrepancy across the spatial interval I. Because the functions ϕₙ(x) are strictly orthonormal, the integral of the cross-terms vanishes identically, leaving only the sum of the squared coefficients within each block:
Step 4: Final Convergence via the Borel–Cantelli Lemma. We sum this bound across all dyadic tiers k from 0 to ∞:
- ,li>
The Standard 𝔏² Energy Test:
-
The Rademacher–Menshov Test:
\[ \sum\,\left\vert c_n \right\vert^2 \, \ln^2 (1+n)} = \sum\,\frac{1}{n^2} = \frac{\pi^2}{6} < \infty . \]
The following Mathematica script defines this threshold system using a high-frequency, staggered trigonometric system to simulate an arbitrary orthogonal framework and demonstrates its stable convergence
On an infinite horizon, a function can easily satisfy the local Dirichlet conditions (being piecewise smooth and of bounded variation locally) but **fail to converge pointwise** at the tails. Because the domain stretches to infinity, the eigenfunctions grow or oscillate rapidly at the horizons.
To guarantee pointwise mean-value convergence for unbounded systems, the function \(f(x)\) must satisfy strict **global asymptotic decay conditions**. For instance, in Hermite polynomial expansions, the function must decay faster than the weight metric: \[ \int_{-\infty}^{\infty} |f(x)|^2 e^{x^2/2} \, {\text d}x < \infty . \] If a function's tails do not drop off rapidly enough at infinity, the expansion series will diverge violently near the horizons, completely breaking the standard Dirichlet metric expansion pipeline.
Example 15: We consider a perfectly square-integrable over the infinite interval (so its -energy is finite and it belongs to the Hilbert space), but possesses a localized geometric defect that causes its orthogonal approximation series to diverge pointwise at a specific coordinate.
A classic, highly illustrative choice on the half-line [0, ∞) under the standard Laguerre weight w(x) = e−x is a localized shifted power function with a sharp singularity at the origin: \[ f(x) = x^{-1/4} \, e^{-x/2} . \] Its global energy norm converges beautifully on the infinite interval: \[ \| f(x) \|^2 = \int_0^{\infty}\, \left\vert \frac{e^{-x/2}}{x^{1/4}} \right\vert^2 \,e^{-x}\,{\text d}x = \int_0^{\infty}\, \frac{e^{-2x}}{x^{1/2}}\, {\text d}x = \sqrt{\frac{\pi}{2}} < \infty . \] Because f(x → ∞ as x → 0+0, it violates the smoothness domain criteria at the left boundary. While the 𝔏² integrated error over the whole infinite interval shrinks to zero, the pointwise polynomial approximations will violently oscillate and shoot to infinity at x = 0 as you increase the number of terms, illustrating absolute local divergence.
To establish a completely rigorous baseline, we must present the Spectral Theorem for Unbounded Self-Adjoint Operators in its exact operator-theoretic format. Because a general unbounded operator cannot be expressed as a single matrix, its spectral decomposition requires lifting the classical eigenvalues into a continuous projection system known as a Projection-Valued Measure (PVM) or Spectral Measure.
Below is the formal theorem statement, followed by a complete analytical proof using the classical Cayley Transform, which maps dangerous unbounded operators into safe, bounded unitary operators.
Theorem 10 (Spectral Theorem for Unbounded Self-Adjoint Operators): Let ℌ be a complex Hilbert space, and let ℒ : 𝐷(ℒ) ⊂ ℌ → ℌ be an unbounded, densely defined, self-adjoint linear operator. There exists a unique Projection-Valued Measure E defined on the Borel σ-algebra ℬ(ℝ) of the real line, acting on ℌ, such that:
-
Domain Condition: The domain is completely characterized by the spectral energy constraint:
\[ \mathcal{D}(\mathcal{L}) = \left\{ f \in \mathcal{H} \;\middle\vert{}\; \int_{-\infty}^{\infty} \lambda^2 \, {\text d}\langle E(\lambda)f, f \rangle < \infty \right\} . \]
-
Integral Representation: For any vector
, the operator evaluates exactly to the spectral spectral integral:
\[ \mathcal{L}\,f = \int_{-\infty}^{\infty} \, \lambda\,{\text d}E(\lambda )\, f . \]
Step 1: Defining the Cayley Transform. Because ℒ is self-adjoint, its eigenvalues must be strictly real. This implies that the complex shifts ℒ + ⅉI and ℒ − ⅉI are strictly invertible mappings from 𝐷(ℒ) onto the entire Hilbert space ℌ.
. We define the Cayley Transform U of the operator ℒ as:
- The domain of U is the entire Hilbert space ℌ. .
-
Isometry: For any element g = (ℒ + ⅉI) f, we have:
\[ \| U\,g \|^2 = \| \left( \mathcal{L} - \mathbf{j}\, I \right) f \|^2 = \| \mathcal{L}\,f \|^2 + \| f \|^2 = \| \left( \mathcal{L} + \mathbf{j}\, I \right) f \|^2 = \| g \|^2 . \]
Step 2: Extracting the Unitary Spectral Decomposition. Because U is a bounded unitary operator, we can directly apply the standard Spectral Theorem for Bounded Unitary Operators. This guarantees the existence of a unique projection-valued measure E supported strictly on the unit circle in the complex plane, allowing us to write:
Step 3: Inverting the Transform back to ℒ . We can algebraically invert our original definition of the Cayley Transform to express our unbounded operator ℒ directly in terms of the bounded unitary operator U:
Step 4: Transforming Coordinates to the Real Line. We perform a trigonometric change of variables by mapping the coordinates along the unit circle back onto the real axis. Let:
By defining a new projection-valued measure on the real line via this mapping, \( \displaystyle \quad \tilde{E} (\Delta ) = E \left( \left\{ \theta \ : \ \lambda (\theta ) = \Delta \right\} \right) ,\quad \) the transformation yields the final spectral representation:
By definition, any finite truncation of a polynomial expansion:
Where Do Orthogonal Functions Come From?
There are significant differences between the behavior of Fourier- and power-series expansions. A power series is essentially an expansion about a point, using only information from that point about the function to be expanded (including, of course, the values of its derivatives). We already know that such expansions only converge within a radius of convergence defined by the position of the nearest singularity. However, a Fourier series (or any expansion in orthogonal functions) uses information from the entire expansion interval, and therefore can describe functions that have “nonpathological” singularities within that interval. However, we also know that the representation of a function by an orthogonal expansion is only guaranteed to converge in the mean. This feature comes into play for the expansion of functions with discontinuities, where there is no unique value to which the expansion must converge. However, for Fourier series, it can be shown that if a function f(x) satisfying the Dirichlet conditions is discontinuous at a point x0, its Fourier series evaluated at that point will be the arithmetic average of the limits of the left and right approaches.
It is most natural to consider pointwise convergence, where we examine the discrepancy at every point. It is difficult (may be impossible) to provide general results about pointwise convergence for general orthogonal functions, but for Fourier series a lot has been done. We have to ask that f(x) be a lot smoother than just belonging to 𝔏², which is a "big" space of functions. You might think having f(x) continuous would be enough, but that obviously does not work at Gibbs phenomenon shows. Here is a slightly different statement of the conditions copied from Keener:
If function f(x) has continuous first derivatives on the interval, except possibly at a finite number of points at which there is a jump in f(x), where left and right derivatives must exist. Then the Fourier series of f converges to ½(f(x+)+f(x−)) for every point in the open interval (0,ℓ). At x=0 and ℓ, the series converges to ½(f(0+)+f(ℓ−)).
Another form of convergence is uniform convergence. This used to be the gold standard of convergence. For continuous functions, you can measure the maximum difference between the series and the function written: