From the theory of ODEs, we know that an effective way to keep track of a seemingly untameable system is through a potential function (a.k.a. a Lyapunov function, or more simply a monovariant). We’ll see two examples from learning theory that uses a potential function inspired by statistical physics.

$\providecommand\opnorm[1]{|#1|_\mathrm{op}}$

$\providecommand\eps\varepsilon\providecommand\E{\mathbb E} \providecommand\GAP{\mathsf{GAP}} \providecommand\tr{\mathop{\mathrm{tr}}} \providecommand{\diag}{\mathop{\mathrm{diag}}} \providecommand\norm[1]{\left|#1\right|} \providecommand{\Var}{\mathrm{Var}} \providecommand\lrangle[1]{\langle#1\rangle}$

In a previous blogpost I proved a mean-field gap bound for PSD matrices in the high temperature regime that has the correct dependence on the temperature $O(\beta^2)$. I recently revisited this problem and found a much better tool to use that allows us to not only do this for all matrices $J$ but also deal with an external field $h$.

$\providecommand{\E}{\mathbb E} \providecommand{\Var}{\mathrm{Var}}\providecommand{\I}{\mathbb I} \providecommand{\Cov}{\mathrm{Cov}}$ I encountered this fact while solving a problem and I was a little shocked that (1) I had never seen this fact before and (2) I had trouble proving it:

Let $Z \sim \mathcal N(0,1)$. Then $\mathrm{Var}(Z | Z < c) \le \mathrm{Var}(Z)$.

(If you think this fact is obvious, you might want to check if your intuition suggests that this is true for all distributions, which it is not: if you have a support of three values $a,b,b+\epsilon$ with $p(a)=p(b) = \delta$ small, then realize that conditioning on being less than $b+\epsilon$ actually increases the variance.)

A while ago I wrote a blogpost with this problem, and I fakesolved it! So here’s my redeeming attempt (and thanks to Zhao Yu for pointing it out to me).

Let $f:\mathbb R \to \mathbb R$ be an infinitely differentiable function. Show that if each point has a derivative that is 0, then $f$ is a polynomial.

This problem was given in the context of applications of the “Bear Cat theorem”, which says something like:

(Folklore) Consider the function sending a polygon to the polygon formed by its midpoints. Show that if we iterate this function, the limiting shape (rescaled appropriately) is an affine transform of a regular polygon.

Sketch:

• Write the explicit form out using root-of-unity filter.
• Consider every $2n$-th term (where $n$ is the number of sides of the polygon). Then, there is a clear leading order term.
• This gives us the explicit form of the “limiting shape”, which is some linear transform of the original polygon.
• Finally, if $a_i$ is the resulting vertex then we can note some relation like $a_{i-1} +a_{i+1} = c\cdot a_i$, and we can check that this $c$ is the same as if $a_i$’s formed a regular polygon.

Remarks. It’s also possible to consider the transform as a linear map on $\mathbb C^n$, then one can diagonalize that matrix to get that the limiting shape comes from the second eigenvector.