Blog

I like playing with interesting ideas and do some analysis on toy problems (like quadratics).

1. Non-asymptotic local convergence analysis of alternating minimization.
   Wenzhi Gao. 04/12/2026.
   HTML  |  PDF
   We provide a proof template for obtaining non-asymptotic local linear convergence rates of two-block alternating minimization (AM). Unlike traditional Jacobian-based asymptotic arguments, we directly show convergence of the function value gap, achieving equally tight but non-asymptotic guarantees.
2. Online Learning to Precondition I. Space dilation methods for linear systems.
   Wenzhi Gao. 03/16/2026.
   HTML  |  PDF
   This is the beginning of a series of posts on Online Learning to Precondition (OL2P): learning to improve the optimization landscape through the behavior of algorithms. As the starting post, we derive a new algorithm for solving linear systems that achieves superlinear convergence, using a space dilation principle motivated by online learning.
3. Hypergradient acceleration.
   Wenzhi Gao (with Yifa Yu). 03/06/2026.
   HTML  |  PDF  |  Code
   This post studies hypergradient descent (HDM), a stepsize adaptation heuristic for gradient-based methods. We show that when the objective function becomes flat around the optimum, HDM can automatically achieve accelerated convergence rate.
4. Negative stepsizes also make multi-block ADMM converge.
   Wenzhi Gao. 03/02/2026.
   HTML  |  PDF  |  Code (divergence)  |  Code (negative stepsize)
   Inspired by a recent result that negative stepsizes make gradient descent ascent converge on bilinear counterexamples, this post shows that periodically using a negative dual stepsize also fixes the divergence of multi-block ADMM on classical quadratic counterexamples.

AI-assisted results

The following results were developed with AI assistance with accompanying Lean projects.

1. An $\Omega(T^{-3/2})$ lower bound for heavy ball.
   GPT-6 Astra, prompted by Wenzhi Gao. 09/09/2026.
   PDF  |  LEAN project (.zip)
   This result proposes a worst-case last-iterate lower bound of $\Omega(LR^2/T^{3/2})$ for every predetermined heavy-ball schedule with nonnegative stepsizes and momenta in $[0,1)$, using a static Huber chain construction.
2. An $\Omega(\sqrt{n})$ lower bound for infeasible-start long-step interior-point methods.
   GPT-6 Astra, prompted by Wenzhi Gao and Huikang Liu. 09/05/2026.
   PDF  |  LEAN project (.zip)
   This result proposes an explicit reverse-Helmert linear program yielding an $\Omega(\min\{n,\sqrt{n}\log(1/\varepsilon)\})$ lower bound on Newton-direction calls for the specified class of scalar-centered infeasible-start long-step methods.