A Review of Introductory PDE Theory
Huh...
I was motivated to make this note because I feel like there is much more to PDEs than what I mastered in the semester I took the intro course. Also, I want my study of PDEs in this sketch to contain many many more highly-cool pictures than a book like Evans, which I felt could have used many more pictures (although actually in hindsight it did have quite a few). Finally, I want this summary to supplement the material covered in many PDEs courses by giving examples of the application of the theory of PDEs to study things in
- Quantum Mechanics
- Differential and Complex Geometry
- Fluid Mechanics
- Numerics
Most physical phenomena can be modeled very nearly by a second-order differential equation.I honestly might be remembering it a bit (or completely) wrong, but we can maybe try in this sketch to see if we can reconstruct his message from my painfully bad memory. 😃😜- May or may not have been said by Prof. Benjamin Safdi
Date Started: June 25, 2024
Date Finished: June 14, 2025 (Projected, took a while, oops)
Date Released to Public: Unpublished
Table of Contents
- Examples and Classification of PDEs
- Method of Characteristics
- Distributions and Fourier Analysis
- Functional Analysis and Quantum Mechanics
- Optimization $\cap$ Differential Geometry
- Hodge Decomposition
- Fluid Dynamics
- Finite Differences, Finite Elements, and Galerkin Methods
- Wavelet-Based Numerical Methods
- All of Physics is Just Expanding to Second Order
- Discretizations That Show Up in Time Series Modeling
- References
Examples and Classification of PDEs
This survey by Lawrence C. Evans was particularly helpful for me in understanding how mathematicians classify PDEs. There are only a few main methods of classification:
- Order
- The order of a PDE is the highest derivative that appears in the equation.
- Type (Elliptic, Parabolic, Hyperbolic)
- This classification system is particularly for second-order PDEs, in parallel to the classification of conic sections into ellipses, parabolas, and hyperbolas. Just as for conic sections, the classification of PDEs is based on the discriminant of the equation.
- The above point is only mostly true, since those who study PDEs tend to use the terms much more losely, on the basis of how the solutions behave, rather than thinking about the discriminant.
- The Cahn-Hilliard equation, for example, is considered parabolic, despite not really being a second-order PDE (something that confused me for a while).
- Linearity
- A PDE is linear if it is linear in the unknown function and its derivatives. Formally, if it can be written as $L(u) = f$ where $L$ is a linear differential operator, then the PDE is linear.
- Similar classifications exist for quasilinear, semilinear, and nonlinear PDEs, although these terms can get a bit confusing until used frequently in practice.
- Linearity does matter in terms of what kinds of behaviors to expect and what techniques can be applied to the PDE.
Some examples of PDEs include the transport equation, the heat equation, the wave equation, the Laplace equation, the Poisson equation, the Schrödinger equation, the Navier-Stokes equations, the Euler equations, Maxwell’s equations, the biharmonic equation, and the Hamilton-Jacobi equations. A good reference for a list of important PDEs is in the first chapter of Evan’s book, as well as the survey linked above.
Method of Characteristics
The method of characteristics is actually a pretty simple idea. Consider an equation like this:
These lines perpendicular to $\begin{bmatrix} 1 \\ b \end{bmatrix}$ are known as characteristics. We have already seen how they can solve the homogeneous transport equation with constant coefficients. But they are far more powerful.
Firstly, characteristics will still work even if we remove the need for homogeneity. Suppose we instead had
Woah woah woah. That strat is jumping ahead a little bit. Can we really solve arbitrary (even nonlinear) first order PDEs with the method of characteristics we've been developing above?
Distributions and Fourier Analysis
As Strichartz put it in his amazing book, sometimes it doesn't make so much sense to worry about individual points $x$. Read the first few pages of his book for a more thorough explanation, but the idea is that we can better understand these functions by "testing" them with test functions. Questions like these lead to the notion of distributions.
The applicability of Fourier analysis to PDEs comes down to the idea that differentiation is a linear operator which corresponds to a convolution with a particular distribution!
One constructive line of reasoning is that the derivative corresponds to convolution with the derivative of the Dirac Delta distribution, which can be seen via integration by parts. A more algebraic line of reasoning is that operators which commute with translations are convolutions, and differentiation indeed commutes with translations (by chain rule).
One major property of Fourier analysis is that it makes convolutions easy by converting them into products. This is a very powerful property, and it is exactly what makes it so useful when we have differential equations involving constant coefficient differential operators.
Functional Analysis and Quantum Mechanics
Optimization $\cap$ Differential Geometry
The study of differential equations is intimately connected to geometry through the idea of variational calculus. In fact, I claim that there is a deep sort of correspondence between variational problems and differential equations.
To see this, we can start with the Laplace equation.
Hodge Decomposition
Fluid Dynamics
Finite Differences, Finite Elements, and Galerkin Methods
The translation of the study of differential equations into the discrete setting is not very difficult, and mainly consists of replacing the differential operator with a difference operator. However, there are many different ways to do this, and each has its own advantages and disadvantages, leading to different numerical schemes for the same differential equation.
Choosing a mesh size $h$, there are two main ways to replace the differential operator: a forward difference or a backward difference. $$D^+_x f(x) = \frac{1}{h}(f(x+h) - f(x))$$ $$D^-_x f(x) = \frac{1}{h}(f(x) - f(x-h))$$ Often in the discrete setting, we set $x_n = nh + x_0$ for some $x_0$ and $h$ (the mesh or step size), and then discretize the function as $f_n = f(x_n)$ (or in more clever way if it suits the problem). This way: $$D^+_x f_n = \frac{1}{h}(f_{n+1} - f_n)$$ $$D^-_x f_n = \frac{1}{h}(f_n - f_{n-1})$$ Of course, a natural question to ask is whether or not this replacement is justified. The main tool we can use for justification is Taylor's theorem: $$f(x+h) = f(x) + f'(x)h + \frac{f''(x)}{2}h^2 + \frac{f'''(x)}{6}h^3 + \cdots$$ so that $$D^+_x f(x) = \frac{1}{h}(f(x+h) - f(x)) = f'(x) + O(h^2)$$ so that for sufficiently small mesh size $h$, the difference between the forward difference and the derivative is small.
While I am still thinking about this question and need to get more opinions, I think that the answer is that the continuous setting can be seen as more clean. There are many discretizations that could correspond to one continuous problem up to very small errors, and there are tools for understanding the nature, existence, and uniqueness of solutions to the continuous problem that may not be available for the discrete setting. Not only that, discrete systems feel more rigid in many cases and blow up more easily (needing more stability conditions based on mesh sizes and time step sizes), so it is nice to have a continuous setting where we can develop qualitative expectations for our numerics.<br/><br/>
Overall though, I feel that it is a bit of a false dichotomy to think of the continuous setting as the "true" setting and the discrete setting as the "approximate" setting. I think that the continuous setting is a very powerful tool for understanding the nature of the problem, and the discrete setting is a very powerful tool for actually computing solutions to the problem. At the end of the day, the practicallity of the model is the error between the discrete scheme and the true phenomenon, which is at most the sum of the error of the numerical scheme with respect to the continuous model and the error of the continuous model itself—so in a machine learning sense, penalizing the error of the numerical scheme with respect to the continuous model is really like regularizing to match our qualitative expectations which are meant to be encapsulated in the continuous model.<br/><br/>
This philosophical approach to modeling seems to take the side of interpretability over pure pragmatics, so I believe it contributes a lot to the debate about the importance of interpretability in modeling. On the other hand, I think that the black box approaches of ML have seen enormous success recently, which has led me to the painful question of how <i>much</i> interpretability is really important in modeling.
</div>
Wavelet-Based Numerical Methods
All of Physics is Just Expanding to Second Order
Discretizations That Show Up in Time Series Modeling
I'm pained to admit that I forgot to write something down about this here in a comment, so what I meant by this may be lost to history since I'm coming back to finish it a year later.... I'll try to recover what I meant by this though, based on other notes in my personal diary that tell me what I was thinking about at the time.
References
- Evans PDE book
- Evans PDE survey
- Strichartz Distributions Book
- Tveito and Winther Intro to PDEs book