Just penning down a note about how establishing that the gap is $O(\beta^2)$ is inherently related to subgaussianity.

$\newcommand{\E}{\mathbb E}$ $\newcommand{\lrangle}[1]{\left\langle #1 \right\rangle}$ $\newcommand{\Var}{\mathrm{Var}}$

In essence, the mean-field problem is about providing a reverse bound to the inequality

$$\E e^{f(X)} \ge \exp \E f(X)$$

In the Gibbs’ context, this distribution isn’t necessarily the uniform distribution over states: given a temperature-invariant field term $\lrangle{h,\sigma}$, we may opt instead to take the distribution given by $\propto e^{\lrangle{h,\sigma}}$.

One naive thing to try is the following: we take $N$ copies and multiply them together: $$ \begin{align*} [\E e^{f(X)}]^N &= \E e^{f(X_1)+f(X_2)+...+f(X_N)} \\ & \approx \E \exp(N\E f(X) + Z\cdot (N \Var(f(X)))^{1/2})\\ & = \exp(N (\E f(X) + \Var f(X))) \end{align*} $$ where $Z\sim \mathcal N(0,1)$ is a normal variable, and here we’re using the CLT for summing $f(X)$’s.

Unfortunately, this doesn’t work. The upper tail of $f(X)$ influences the sum too much. For instance, the upper tail $P(X\le t)$ needs to decay as fast as $e^{-ct}$.

All of these suggest the following. Suppose we want to study the partition function of $$f(\sigma) = \frac \beta 2 \sigma^\top J \sigma + \lrangle{h, \sigma}$$ We reweigh the distribution of $\sigma$ to follow the distribution with mean $\tilde h = \tanh h$, so

$$ \begin{align*} Z &= \sum_\sigma \exp\left(\frac \beta 2 \sigma^\top J \sigma + \lrangle{h, \sigma}\right) \\ &= Z_{\tilde h} \E_{\tilde h} \exp\left(\frac \beta 2 \sigma^\top J \sigma\right) \end{align*} $$ and we would like an upper bound like $$\E_{\tilde h} \exp\left(\frac \beta 2 \sigma^\top J \sigma\right) \overset ? \le \exp\left( \frac \beta 2 \E_{\tilde h} \sigma^\top J \sigma + C \beta^2 \right)$$ where the first term is the mean-field approximation, so the “gap” is $C\beta^2$. What this really means is that $\sigma^\top J \sigma$ under $\sigma\sim \xi(\tilde h)$ is subgaussian with constant $C$.