Advanced Differential Equations - Series Solutions, Laplace Transforms, and Fourier Analysis

While many fundamental physical systems can be described by ordinary differential equations (ODEs) with constant coefficients, a vast number of problems in science and engineering involve more complex scenarios—variable coefficients, discontinuous forcing terms, or systems described by their boundary conditions. To tackle these, mathematicians have developed a powerful arsenal of advanced techniques. This exploration provides an in-depth guide to three such pillars of applied mathematics: series solutions, Laplace transforms, and Fourier analysis, concluding with an introduction to the realm of partial differential equations (PDEs).

---

Series Solutions of ODEs - When Elementary Functions Fail

For many ODEs with variable coefficients, solutions cannot be expressed in terms of elementary functions (polynomials, exponentials, logarithms, etc.). In these cases, we seek a solution in the form of an infinite power series.

Power Series Method

This method is applicable when the coefficients of the ODE are analytic. We assume a solution of the form: $y(x) = \sum_{n=0}^\infty a_n (x-x_0)^n$ By substituting this series into the differential equation and equating coefficients of like powers of $x$, we can determine a recurrence relation for the coefficients $a_n$.

Legendre’s Equation and Legendre Polynomials

A key example is Legendre’s Equation: $(1-x^2)y'' - 2xy' + \alpha(\alpha+1)y = 0$ where $\alpha$ is a constant. This equation arises in numerous physics problems involving spherical symmetry, such as solving for the electrostatic potential or the Schrödinger equation for the hydrogen atom. When the power series method is applied, it is found that if $\alpha$ is a non-negative integer, $\alpha=n$, one of the series solutions terminates, resulting in a polynomial of degree $n$. These polynomial solutions are known as the Legendre Polynomials, $P_n(x)$.

$P_n(x) = \dfrac{1}{2^n} \sum_{k=0}^{\left\lfloor \dfrac{n}{2} \right\rfloor} \dfrac{(-1)^k (2n - 2k)!}{k!(n - k)!(n - 2k)!} x^{n - 2k}$ $Q_n(x) = \dfrac{1}{2} P_n(x) \ln\left(\dfrac{1+x}{1-x}\right) + R_n(x)$ where $R_n(x)$ is a polynomial of degree $n-1$ that compensates for the non-polynomial part. The general solution is a linear combination of $P_n(x)$ and $Q_n(x)$. In bounded solutions, we typically only consider $P_n(x)$.

The first few Legendre Polynomials are:

• $P_0(x) = 1$
• $P_1(x) = x$
• $P_2(x) = \dfrac{1}{2}(3x^2 - 1)$
• $P_3(x) = \dfrac{1}{2}(5x^3 - 3x)$

Extended Power Series Method: Frobenius Method

When an ODE has a regular singular point at $x_0$, the standard power series method may fail. The Frobenius method extends it by seeking a solution of the form: $y(x) = x^r \sum_{n=0}^\infty a_n x^n = \sum_{n=0}^\infty a_n x^{n+r}$ where $r$ is a constant (which can be a non-integer) that is determined by solving an indicial equation derived from the ODE.

Bessel’s Equation and Bessel Functions

Another crucial equation in physics and engineering is Bessel’s Equation: $x^2y'' + xy' + (x^2-\nu^2)y = 0$ where $\nu$ is a constant. This equation appears in problems involving vibrations of a circular membrane, heat conduction in a cylinder, and electromagnetic waves in cylindrical waveguides. Since $x=0$ is a regular singular point, the Frobenius method is used to find its solutions.

• Bessel Functions of the First Kind, $J_\nu(x)$: These are the solutions that are finite at $x=0$. $J_\nu(x) = \sum_{k=0}^\infty \dfrac{(-1)^k (x/2)^{\nu + 2k}}{k! \Gamma(k + \nu + 1)}$ where $\Gamma$ is the gamma function. When $k + \nu + 1$ is a non-negative integer, $\Gamma(k + \nu + 1) = (k + \nu)!$. $\Gamma$ function is defined by: $\Gamma(z) = \int_0^\infty t^{z-1} e^{-t} dt$ Properties of Bessel Functions:
  • $J_{-\nu}(x) = (-1)^\nu J_\nu(x)$
  • $(x^{\nu} J_\nu(x))' = x^{\nu} J_{\nu-1}(x)$
  • $(x^{-\nu} J_\nu(x))' = -x^{-\nu} J_{\nu+1}(x)$
  • $J_{\nu-1}(x) + J_{\nu+1}(x) = \dfrac{2\nu}{x} J_\nu(x)$
  • $J_{\nu-1}(x) - J_{\nu+1}(x) = 2 J_\nu'(x)$
  • $J_{1/2}(x) = \sqrt{\dfrac{2}{\pi x}} \sin(x)$
  • $J_{-1/2}(x) = \sqrt{\dfrac{2}{\pi x}} \cos(x)$
  • $J_0(x) = \sum_{k=0}^\infty \dfrac{(-1)^k (x/2)^{2k}}{(k!)^2}$, which converges for all $x$.
  • $J_1(x) = \sum_{k=0}^\infty \dfrac{(-1)^k (x/2)^{2k+1}}{(k+1)!k!}$, which converges for all $x$.
  • $J_n(x) = \dfrac{1}{\pi} \int_0^\pi \cos(n\theta - x\sin\theta) d\theta$.
  • $J_n(x)$ is oscillatory for large $x$ and decays as $x^{-1/2}$.
• Bessel Functions of the Second Kind, $Y_\nu(x)$ (or Neumann Functions): When $\nu$ is not an integer, $J_{-\nu}(x)$ provides a second, linearly independent solution. When $\nu$ is an integer, the second solution, $Y_\nu(x)$, is singular (blows up) at $x=0$. $Y_\nu(x) = \dfrac{J_\nu(x) \cos(\nu \pi) - J_{-\nu}(x)}{\sin(\nu \pi)}$ For integer $\nu$, $Y_\nu(x)$ is defined using a limit: $Y_n(x) = \lim_{\nu \to n} Y_\nu(x)$ $Y_\nu(x) = \dfrac{1}{\pi} \int_0^\pi \sin\left( x \sin \theta - \nu \theta \right) d\theta - \dfrac{1}{\pi} \int_0^\infty \left( e^{\nu t} + e^{-\nu t} \right) e^{-x \sinh t} dt$ Properties of Bessel Functions of the Second Kind:
  • $Y_{-\nu}(x) = (-1)^\nu Y_\nu(x)$
  • $Y_0(x) = -\dfrac{2}{\pi} \ln\left(\dfrac{x}{2}\right) J_0(x) + \text{finite terms}$
  • $Y_1(x) = -\dfrac{2}{\pi} \ln\left(\dfrac{x}{2}\right) J_1(x) + \text{finite terms}$
• General Solution: The general solution to Bessel's equation is a linear combination of these two types of functions: $y(x) = c_1 J_\nu(x) + c_2 Y_\nu(x)$

---

Laplace Transforms - Turning Calculus into Algebra

The Laplace transform is a powerful integral transform used to solve linear ODEs and integral equations. It excels at handling initial value problems and discontinuous forcing functions.

• Laplace Transform Definition: The Laplace transform of a function $f(t)$, defined for $t \ge 0$, is the function $F(s)$ given by: $\mathcal{L}\{f(t)\} = F(s) = \int_0^\infty e^{-st} f(t) dt$
• Linearity: The transform is linear: $\mathcal{L}\{a f(t) + b g(t)\} = a F(s) + b G(s)$.
• f periodic with p: $\mathcal{L}(f)=\dfrac{1}{1-e^{-ps}}\int_0^p e^{-st}f(t)dt$

Key Properties and Theorems

1. First Shifting Theorem (s-Shifting): This theorem relates the transform of an exponential-multiplied function to the transform of the original function. $\mathcal{L}\{e^{at}f(t)\} = F(s-a)$ Derivation: By definition, $\mathcal{L}\{e^{at}f(t)\} = \int_0^\infty e^{-st} (e^{at}f(t)) dt = \int_0^\infty e^{-(s-a)t} f(t) dt$. This is precisely the definition of $F(s)$ with $s$ replaced by $s-a$, so it is $F(s-a)$.

2. Transforms of Derivatives and Integrals: This is the property that makes the Laplace transform so useful for solving ODEs.

   • Derivative: $\mathcal{L}\{f'(t)\} = sF(s) - f(0)$ $\mathcal{L}\{f''(t)\} = s^2F(s) - sf(0) - f'(0)$ $\mathcal{L}\{f^{(n)}(t)\} = s^nF(s) - s^{n-1}f(0) - s^{n-2}f'(0) - \ldots - f^{(n-1)}(0)$
   • Derivation of $\mathcal{L}\{f'(t)\}$: Using integration by parts on the definition $\int_0^\infty e^{-st}f'(t)dt$. Let $u=e^{-st}$ and $dv=f'(t)dt$. Then $du=-se^{-st}dt$ and $v=f(t)$. $\mathcal{L}\{f'(t)\} = [e^{-st}f(t)]_0^\infty - \int_0^\infty f(t)(-se^{-st})dt = (0 - f(0)) + s \int_0^\infty e^{-st}f(t)dt = -f(0) + sF(s)$.
   • Integral: $\mathcal{L}\left\{\int_0^t f(\tau)d\tau\right\} = \dfrac{F(s)}{s}$.

3. Unit Step Function (Heaviside Function) and Second Shifting Theorem (t-Shifting):

   • The unit step function, $u(t-a)$, is 0 for $t<a$ and 1 for $t \ge a$. It models functions that are "switched on" at a certain time.
   • Second Shifting Theorem: This theorem is used to find the transform of a function that is shifted in time. $\mathcal{L}\{f(t-a)u(t-a)\} = e^{-as}F(s)$

4. Short Impulses and Dirac’s Delta Function:

   • The Dirac delta function, $\delta(t-a)$, models an infinitely high, infinitesimally narrow spike at $t=a$ with a total area of 1. It represents an impulse.
   • Its Laplace transform is $\mathcal{L}\{\delta(t-a)\} = e^{-as}$.

5. Convolution: The convolution of two functions $f(t)$ and $g(t)$ is defined as: $(f*g)(t) = \int_0^t f(\tau)g(t-\tau)d\tau$

   • Convolution Theorem: The Laplace transform of a convolution is the product of the individual Laplace transforms. $\mathcal{L}\{f*g\} = F(s)G(s)$ This theorem is extremely useful for finding inverse transforms of products and for solving integral equations.

6. Differentiation and Integration of Transforms: These properties can be used to find new transforms and solve ODEs with variable coefficients.

   • $\mathcal{L}\{tf(t)\} = -F'(s)$
   • $\mathcal{L}\left\{\dfrac{f(t)}{t}\right\} = \int_s^\infty F(\tilde{s})d\tilde{s}$

To use the Laplace transform method, one typically consults tables of general formulas and transforms.

Table of Common Laplace Transforms

$f(t)$	$F(s) = \mathcal{L}\{f(t)\}$
$1$	$\dfrac{1}{s}$
$t^n$	$\dfrac{n!}{s^{n+1}}$
$\dfrac{t^{n-1}}{(n-1)!}$	$\dfrac{1}{s^n}$ (for $n = 1, 2, \ldots$)
$\dfrac{t^{a-1}}{\Gamma(a)}$	$\dfrac{1}{s^a}$ (for $a \gt 0$)
$t^a$	$\dfrac{\Gamma(a+1)}{s^{a+1}}$ (for $a > -1$)
$e^{at}$	$\dfrac{1}{s-a}$ (for $s > a$)
$\dfrac{t^{n-1}e^{at}}{(n-1)!}$	$\dfrac{1}{(s-a)^n}$ (for $n = 1, 2, \ldots$)
$\dfrac{t^{k-1}e^{at}}{\Gamma(k)}$	$\dfrac{1}{(s-a)^k}$ (for $k \gt 0$)
$\sin(\omega t)$	$\dfrac{\omega}{s^2 + \omega^2}$
$\cos(\omega t)$	$\dfrac{s}{s^2 + \omega^2}$
$\sinh(\omega t)$	$\dfrac{\omega}{s^2 - \omega^2}$
$\cosh(\omega t)$	$\dfrac{s}{s^2 - \omega^2}$
$e^{at} \sin(\omega t)$	$\dfrac{\omega}{(s-a)^2 + \omega^2}$
$e^{at} \cos(\omega t)$	$\dfrac{s-a}{(s-a)^2 + \omega^2}$

---

Fourier Analysis - Decomposing into Harmonics

Fourier analysis is a powerful branch of mathematics for decomposing a function into a sum (or integral) of simple sine and cosine waves. It is essential for signal processing, vibration analysis, and solving PDEs.

Fourier Series

A Fourier series represents a periodic function $f(x)$ with period $2L$ as an infinite sum of sines and cosines: $f(x) = a_0 + \sum_{n=1}^\infty \left( a_n \cos\left(\dfrac{n\pi x}{L}\right) + b_n \sin\left(\dfrac{n\pi x}{L}\right) \right)$

• Derivation of Fourier Coefficients: The coefficients are found by exploiting the orthogonality of sine and cosine functions over the interval $[-L, L]$. For example, $\int_{-L}^L \cos(\dfrac{n\pi x}{L})\cos(\dfrac{m\pi x}{L})dx = 0$ if $n \neq m$.
  • To find $a_0$, integrate $f(x)$ from $-L$ to $L$: $a_0 = \dfrac{1}{2L}\int_{-L}^L f(x) dx \quad (\text{the average value of } f(x))$
  • To find $a_n$ for $n \ge 1$, multiply the series by $\cos(\dfrac{m\pi x}{L})$ and integrate from $-L$ to $L$. Due to orthogonality, all terms except the one where $n=m$ become zero. $a_n = \dfrac{1}{L}\int_{-L}^L f(x)\cos\left(\dfrac{n\pi x}{L}\right) dx$
  • To find $b_n$, multiply by $\sin(\dfrac{m\pi x}{L})$ and integrate: $b_n = \dfrac{1}{L}\int_{-L}^L f(x)\sin\left(\dfrac{n\pi x}{L}\right) dx$
• Arbitrary Period, Even/Odd Functions, Half-Range Expansions:
  • If a function has period $T$, replace $L$ with $T/2$.
  • If $f(x)$ is an even function, its Fourier series will only contain cosine terms ($b_n=0$).
  • If $f(x)$ is an odd function, its Fourier series will only contain sine terms ($a_n=0$ for $n \ge 0$).
  • A function defined only on $[0, L]$ can be extended as either an even or odd function to $[-L,L]$ to obtain a half-range cosine or sine series, respectively.
• Orthogonality of the Trigonomtric system:
  • $\int_{-L}^L \cos\left(\dfrac{n\pi x}{L}\right)\cos\left(\dfrac{m\pi x}{L}\right) dx = \begin{cases} L & n = m \\ 0 & n \neq m \end{cases}$
  • $\int_{-L}^L \sin\left(\dfrac{n\pi x}{L}\right)\sin\left(\dfrac{m\pi x}{L}\right) dx = \begin{cases} L & n = m \\ 0 & n \neq m \end{cases}$
  • $\int_{-L}^L \cos\left(\dfrac{n\pi x}{L}\right)\sin\left(\dfrac{m\pi x}{L}\right) dx = 0$ for all $n, m$.
• Convergence of Fourier Series: The Fourier series converges to the function at points where the function is continuous. At points of discontinuity, it converges to the average of the left-hand and right-hand limits (known as the Dirichlet conditions).

Fourier Series in Complex Form

The Fourier series can also be expressed in complex form using Euler's formula: $f(x) = \sum_{n=-\infty}^\infty c_n e^{i\dfrac{n\pi x}{L}}$ where the coefficients are given by: $c_n = \dfrac{1}{2L}\int_{-L}^L f(x)e^{-i\dfrac{n\pi x}{L}} dx$

Sturm–Liouville Problems

This is a general theory of second-order ODEs of the form: $\dfrac{d}{dx}\left[p(x)\dfrac{dy}{dx}\right] + [q(x) + \lambda w(x)]y = 0$ subject to certain boundary conditions.

• Orthogonal Functions: The solutions (eigenfunctions) corresponding to distinct eigenvalues $\lambda$ of a Sturm-Liouville problem are orthogonal with respect to the weight function $w(x)$. $\int_a^b y_m(x) y_n(x) w(x) dx = 0 \quad \text{for } m \neq n$

• Generalized Fourier Series: Any well-behaved function can be expanded as a series of these orthogonal eigenfunctions, known as a generalized Fourier series. Legendre polynomials and Bessel functions are solutions to Sturm-Liouville problems and thus form sets of orthogonal functions. $f(x) = \sum_{n=0}^\infty a_n y_n(x)$

Fourier Integral and Transforms

For non-periodic functions, the idea of a series is extended to a Fourier Integral: $f(x) = \int_0^\infty [A(\omega)\cos(\omega x) + B(\omega)\sin(\omega x)] d\omega$ $A(\omega) = \dfrac{1}{\pi}\int_{-\infty}^\infty f(x)\cos(\omega x) dx$ $B(\omega) = \dfrac{1}{\pi}\int_{-\infty}^\infty f(x)\sin(\omega x) dx$ This leads to the Fourier Transform, which decomposes a function into its continuous frequency spectrum.

• Fourier Transform: $\mathcal{F}\{f(x)\} = \hat{f}(\omega) = \dfrac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty f(x)e^{-i\omega x} dx$.
• Inverse Fourier Transform: $f(x) = \dfrac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \hat{f}(\omega)e^{i\omega x} d\omega$.
• Fourier Cosine and Sine Transforms: Used for even and odd functions, respectively. $\text{Fourier Cosine Transform: } \mathcal{F}_c\{f(x)\} = F_c(\omega) = \sqrt{\dfrac{2}{\pi}} \int_0^\infty f(x)\cos(\omega x) dx$ $\text{Fourier Sine Transform: } \mathcal{F}_s\{f(x)\} = F_s(\omega) = \sqrt{\dfrac{2}{\pi}} \int_0^\infty f(x)\sin(\omega x) dx$
  • Transforms of derivatives:
    • $\mathcal{F}_c\{f'(x)\} = \omega\mathcal{F}_s\{f(x)\} - \sqrt{\dfrac{2}{\pi}}f(0)$
    • $\mathcal{F}_s\{f'(x)\} = -\omega\mathcal{F}_c\{f(x)\}$
    • $\mathcal{F}_c\{f^n(x)\} = -\omega^2\mathcal{F}_c\{f(x)\} - \sqrt{\dfrac{2}{\pi}}f'(0)$
    • $\mathcal{F}_s\{f^n(x)\} = -\omega^2\mathcal{F}_s\{f(x)\} + \sqrt{\dfrac{2}{\pi}}\omega f(0)$
• Discrete and Fast Fourier Transforms (DFT & FFT): These are numerical algorithms for computing the Fourier transform of discrete data, essential for digital signal processing.
• Theorems:
  • Parseval’s Theorem: Relates the integral of the square of a function to the integral of the square of its Fourier transform. $\int_{-\infty}^\infty |f(x)|^2 dx = \int_{-\infty}^\infty |\hat{f}(\omega)|^2 d\omega$
  • Convolution Theorem: The Fourier transform of a convolution is the product of the Fourier transforms. $\mathcal{F}\{f * g\} = \sqrt{2\pi} \mathcal{F}(f) \mathcal{F}(g)$

---

Introduction to Partial Differential Equations (PDEs)

A PDE is a differential equation that contains unknown multivariable functions and their partial derivatives. They model phenomena in higher dimensions, such as wave propagation and heat flow.

Basic Concepts and Solution by Separating Variables

A powerful technique for solving linear PDEs is the method of separation of variables. We assume a solution that is a product of functions, each dependent on only one independent variable. Example: 1D Heat Equation: $\dfrac{\partial u}{\partial t} = k \dfrac{\partial^2 u}{\partial x^2}$.

1. Assume a solution: $u(x,t) = X(x)T(t)$.
2. Substitute into the PDE: $X(x)T'(t) = k X''(x)T(t)$.
3. Separate variables: Divide by $kX(x)T(t)$ to get $\dfrac{T'(t)}{kT(t)} = \dfrac{X''(x)}{X(x)}$.
4. Since the left side depends only on $t$ and the right only on $x$, both must equal a constant, say $-\lambda^2$.
   • $T'(t) + \lambda^2 k T(t) = 0$
   • $X''(x) + \lambda^2 X(x) = 0$
5. Solve the ODEs: These are two separate ODEs that can be solved.
6. Apply Boundary Conditions and Superposition: The solutions for $X(x)$ (often sine and cosine) are combined using boundary conditions, leading to a Fourier series solution for the final answer.

Laplace’s Equation in Different Coordinates

Laplace's Equation, $\nabla^2 u = 0$, describes steady-state potentials (gravitational, electrostatic, temperature). Its form depends on the coordinate system:

• Laplacian in Polar Coordinates: $\nabla^2 u = \dfrac{\partial^2 u}{\partial r^2} + \dfrac{1}{r}\dfrac{\partial u}{\partial r} + \dfrac{1}{r^2}\dfrac{\partial^2 u}{\partial \theta^2}$.

• Laplace’s Equation in Cylindrical and Spherical Coordinates: The Laplacian takes on more complex forms. Solutions often involve Bessel functions (for cylindrical) and Legendre polynomials (for spherical, leading to spherical harmonics). $\text{Cylindrical: } \nabla^2 u = \dfrac{\partial^2 u}{\partial r^2} + \dfrac{1}{r}\dfrac{\partial u}{\partial r} + \dfrac{1}{r^2}\dfrac{\partial^2 u}{\partial \theta^2} + \dfrac{\partial^2 u}{\partial z^2}$ $\text{Spherical: } \nabla^2 u = \dfrac{\partial^2 u}{\partial r^2} + \dfrac{2}{r}\dfrac{\partial u}{\partial r} + \dfrac{1}{r^2}\dfrac{\partial^2 u}{\partial \phi^2} + \dfrac{\cot \phi}{r^2} \dfrac{\partial u}{\partial \phi} + \dfrac{1}{r^2\sin^2\phi} \dfrac{\partial^2u}{\partial \theta^2}$

• Dirichlet and Neumann Boundary Conditions: These specify the value of the function (Dirichlet) or its normal derivative (Neumann) on the boundary of the domain. They are crucial for determining unique solutions to PDEs. $\text{Dirichlet: } u|_{\partial \Omega} = g(x) (\text{function value on boundary})$ $\text{Neumann: } \dfrac{\partial u}{\partial n}\bigg|_{\partial \Omega} = h(x) (\text{normal derivative on boundary})$

Solution of PDEs by Laplace Transforms

The Laplace transform can also be used to solve PDEs by transforming one variable (often time), reducing the PDE to an ODE in the remaining variable(s).

---

Key Takeaways: Advanced Methods for Dynamic Systems

The study of advanced differential equations provides a sophisticated toolkit for tackling a vast range of problems that are inaccessible with elementary methods.

• Series Solutions: The power series and Frobenius methods allow us to find solutions for linear ODEs with variable coefficients, leading to important special functions like Legendre polynomials and Bessel functions that are ubiquitous in physics.
• Laplace Transforms: This powerful integral transform converts differential equations in the time domain into algebraic equations in the frequency domain, providing a streamlined method for solving initial value problems, particularly those involving discontinuous or impulsive inputs.
• Fourier Analysis: Fourier series and transforms are based on the principle of decomposing complex periodic or non-periodic functions into a sum or integral of simple sines and cosines. This is fundamental to signal processing, wave analysis, and solving partial differential equations.
• Partial Differential Equations (PDEs): PDEs are the language used to model multivariable dynamic systems like heat flow, wave motion, and potential fields. Techniques like separation of variables, often in conjunction with Fourier series, are key to finding solutions.
• Unifying Frameworks: Concepts like Sturm-Liouville theory provide a general framework for understanding the orthogonal functions that arise as solutions to many important differential equations, unifying them under a common theoretical structure.

These advanced methods are essential for a deep understanding of physical systems and form the backbone of modern engineering analysis and design.