See all sketches...
cd ~ ..

A Review of Introductory PDE Theory


Huh...

I was motivated to make this note because I feel like there is much more to PDEs than what I mastered in the semester I took the intro course. Also, I want my study of PDEs in this sketch to contain many many more highly-cool pictures than a book like Evans, which I felt could have used many more pictures (although actually in hindsight it did have quite a few). Finally, I want this summary to supplement the material covered in many PDEs courses by giving examples of the application of the theory of PDEs to study things in

(I might slip in some applications of ODEs as well, but don't hate me for it). Finally, I want to talk about a philosophical remark I vaguely remember my physics teacher making in my first semester:
Most physical phenomena can be modeled very nearly by a second-order differential equation.

- May or may not have been said by Prof. Benjamin Safdi

I honestly might be remembering it a bit (or completely) wrong, but we can maybe try in this sketch to see if we can reconstruct his message from my painfully bad memory. 😃😜

Date Started: June 25, 2024
Date Finished: June 14, 2025 (Projected, took a while, oops) Date Released to Public: Unpublished

Table of Contents

  1. Examples and Classification of PDEs
  2. Method of Characteristics
  3. Distributions and Fourier Analysis
  4. Functional Analysis and Quantum Mechanics
  5. Optimization $\cap$ Differential Geometry
  6. Hodge Decomposition
  7. Fluid Dynamics
  8. Finite Differences, Finite Elements, and Galerkin Methods
  9. Wavelet-Based Numerical Methods
  10. All of Physics is Just Expanding to Second Order
  11. Discretizations That Show Up in Time Series Modeling
  12. References

Examples and Classification of PDEs

This survey by Lawrence C. Evans was particularly helpful for me in understanding how mathematicians classify PDEs. There are only a few main methods of classification:

Some examples of PDEs include the transport equation, the heat equation, the wave equation, the Laplace equation, the Poisson equation, the Schrödinger equation, the Navier-Stokes equations, the Euler equations, Maxwell’s equations, the biharmonic equation, and the Hamilton-Jacobi equations. A good reference for a list of important PDEs is in the first chapter of Evan’s book, as well as the survey linked above.

Method of Characteristics

The method of characteristics is actually a pretty simple idea. Consider an equation like this:

$$u_t + b \cdot Du = 0, \qquad t \in \R, x \in \R^n$$
$$u(0,x) = u_0(x), \qquad x \in \R^n$$
where $D$ is the vector of spatial derivatives and $b$ is a constant vector in $\R^n$. Then we can squint our eyes and say:
$$\begin{bmatrix} 1 & b^\top \end{bmatrix} \begin{bmatrix} u_t \\ Du\end{bmatrix} = 0$$
so that our solution $u(t, x)$ is constant along the lines perpendicular to $\begin{bmatrix} 1 \\ b \end{bmatrix}$. In other words, $u(t + s, x + sb) = u(t, x)$ for any $s \in \R$, so if we are given initial data anywhere, we can simply propagate it along these lines via knowledge of our solutions' contours. Here, $u(0, x) = u_0(x)$, so $u(s, x + sb) = u_0(x)$ for any $s \in \R$ and $x \in \R^n$. This gives the solution (after reparametrization):
$$u(t, x) = u_0(x - tb).$$

These lines perpendicular to $\begin{bmatrix} 1 \\ b \end{bmatrix}$ are known as characteristics. We have already seen how they can solve the homogeneous transport equation with constant coefficients. But they are far more powerful.

Firstly, characteristics will still work even if we remove the need for homogeneity. Suppose we instead had

$$u_t + b \cdot Du = f, \qquad t \in \R, x \in \R^n$$
$$u(0,x) = u_0(x), \qquad x \in \R^n$$
so that after squinting,
$$\begin{bmatrix} 1 & b^\top \end{bmatrix} \begin{bmatrix} u_t \\ Du\end{bmatrix} = f.$$
We can actually just take the same characteristics, and now instead of contours, we have ODEs along our characteristics. Define $z(s) = u(t+s, x+ sb)$. Then $$\dot{z}(s) = f$$ is now an ODE.

Strat (Method of Characteristics). First order PDEs can be solved by solving systems of ODEs along characteristics.

Woah woah woah. That strat is jumping ahead a little bit. Can we really solve arbitrary (even nonlinear) first order PDEs with the method of characteristics we've been developing above?

Distributions and Fourier Analysis

As Strichartz put it in his amazing book, sometimes it doesn't make so much sense to worry about individual points $x$. Read the first few pages of his book for a more thorough explanation, but the idea is that we can better understand these functions by "testing" them with test functions. Questions like these lead to the notion of distributions.

The applicability of Fourier analysis to PDEs comes down to the idea that differentiation is a linear operator which corresponds to a convolution with a particular distribution!

Strat (Translation Equivariance of Derivatives). Constant coefficient differential operators are convolutions.

One constructive line of reasoning is that the derivative corresponds to convolution with the derivative of the Dirac Delta distribution, which can be seen via integration by parts. A more algebraic line of reasoning is that operators which commute with translations are convolutions, and differentiation indeed commutes with translations (by chain rule).

One major property of Fourier analysis is that it makes convolutions easy by converting them into products. This is a very powerful property, and it is exactly what makes it so useful when we have differential equations involving constant coefficient differential operators.

Functional Analysis and Quantum Mechanics

Optimization $\cap$ Differential Geometry

The study of differential equations is intimately connected to geometry through the idea of variational calculus. In fact, I claim that there is a deep sort of correspondence between variational problems and differential equations.

To see this, we can start with the Laplace equation.

Hodge Decomposition

Fluid Dynamics

Finite Differences, Finite Elements, and Galerkin Methods

The translation of the study of differential equations into the discrete setting is not very difficult, and mainly consists of replacing the differential operator with a difference operator. However, there are many different ways to do this, and each has its own advantages and disadvantages, leading to different numerical schemes for the same differential equation.

Choosing a mesh size $h$, there are two main ways to replace the differential operator: a forward difference or a backward difference. $$D^+_x f(x) = \frac{1}{h}(f(x+h) - f(x))$$ $$D^-_x f(x) = \frac{1}{h}(f(x) - f(x-h))$$ Often in the discrete setting, we set $x_n = nh + x_0$ for some $x_0$ and $h$ (the mesh or step size), and then discretize the function as $f_n = f(x_n)$ (or in more clever way if it suits the problem). This way: $$D^+_x f_n = \frac{1}{h}(f_{n+1} - f_n)$$ $$D^-_x f_n = \frac{1}{h}(f_n - f_{n-1})$$ Of course, a natural question to ask is whether or not this replacement is justified. The main tool we can use for justification is Taylor's theorem: $$f(x+h) = f(x) + f'(x)h + \frac{f''(x)}{2}h^2 + \frac{f'''(x)}{6}h^3 + \cdots$$ so that $$D^+_x f(x) = \frac{1}{h}(f(x+h) - f(x)) = f'(x) + O(h^2)$$ so that for sufficiently small mesh size $h$, the difference between the forward difference and the derivative is small.

Remark. This is one of the main puzzles I had regarding the study of PDEs. If differential equations were studied as a tool to model physical phenomena, and the discretizations are what we can actually compute, then is there any reason to prefer the continuous setting, rather than just studying and experimenting with the discrete systems?

While I am still thinking about this question and need to get more opinions, I think that the answer is that the continuous setting can be seen as more clean. There are many discretizations that could correspond to one continuous problem up to very small errors, and there are tools for understanding the nature, existence, and uniqueness of solutions to the continuous problem that may not be available for the discrete setting. Not only that, discrete systems feel more rigid in many cases and blow up more easily (needing more stability conditions based on mesh sizes and time step sizes), so it is nice to have a continuous setting where we can develop qualitative expectations for our numerics.<br/><br/>

Overall though, I feel that it is a bit of a false dichotomy to think of the continuous setting as the "true" setting and the discrete setting as the "approximate" setting. I think that the continuous setting is a very powerful tool for understanding the nature of the problem, and the discrete setting is a very powerful tool for actually computing solutions to the problem. At the end of the day, the practicallity of the model is the error between the discrete scheme and the true phenomenon, which is at most the sum of the error of the numerical scheme with respect to the continuous model and the error of the continuous model itself—so in a machine learning sense, penalizing the error of the numerical scheme with respect to the continuous model is really like regularizing to match our qualitative expectations which are meant to be encapsulated in the continuous model.<br/><br/>

This philosophical approach to modeling seems to take the side of interpretability over pure pragmatics, so I believe it contributes a lot to the debate about the importance of interpretability in modeling. On the other hand, I think that the black box approaches of ML have seen enormous success recently, which has led me to the painful question of how <i>much</i> interpretability is really important in modeling.
</div>

Wavelet-Based Numerical Methods

All of Physics is Just Expanding to Second Order

Discretizations That Show Up in Time Series Modeling

I'm pained to admit that I forgot to write something down about this here in a comment, so what I meant by this may be lost to history since I'm coming back to finish it a year later.... I'll try to recover what I meant by this though, based on other notes in my personal diary that tell me what I was thinking about at the time.

References

  1. Evans PDE book
  2. Evans PDE survey
  3. Strichartz Distributions Book
  4. Tveito and Winther Intro to PDEs book