Wenzhi Gao
Home
Research
Papers
Software
Blog
Posts

Blog

I like playing with interesting ideas and do some analysis on toy problems (like quadratics).

  1. Online Learning to Precondition III. Concise analysis of a quasi-Newton type algorithm
    Wenzhi Gao. 10/13/2026.
    HTML  |  PDF
    This short post introduces a low-rank approximate-inverse update through the online learning to precondition viewpoint and analyzes it with a learning potential.
  2. Online Learning to Precondition II. Quasi-Newton methods.
    Wenzhi Gao. 10/13/2026.
    HTML  |  PDF
    This post connects quasi-Newton methods with online learning: their secant equations can be viewed as loss functions for learning the (inverse) Hessian at the optimum.
  3. Non-asymptotic local convergence analysis of alternating minimization.
    Wenzhi Gao. 04/12/2026.
    HTML  |  PDF
    We provide a proof template for obtaining non-asymptotic local linear convergence rates of two-block alternating minimization (AM). Unlike traditional Jacobian-based asymptotic arguments, we directly show convergence of the function value gap, achieving equally tight but non-asymptotic guarantees.
  4. Online Learning to Precondition I. Space dilation methods for linear systems.
    Wenzhi Gao. 03/16/2026.
    HTML  |  PDF
    This is the beginning of a series of posts on Online Learning to Precondition (OL2P): learning to improve the optimization landscape through the behavior of algorithms. As the starting post, we derive a new algorithm for solving linear systems that achieves superlinear convergence, using a space dilation principle motivated by online learning.
  5. Hypergradient acceleration.
    Wenzhi Gao (with Yifa Yu). 03/06/2026.
    HTML  |  PDF  |  Code
    This post studies hypergradient descent (HDM), a stepsize adaptation heuristic for gradient-based methods. We show that when the objective function becomes flat around the optimum, HDM can automatically achieve accelerated convergence rate.
  6. Negative stepsizes also make multi-block ADMM converge.
    Wenzhi Gao. 03/02/2026.
    HTML  |  PDF  |  Code (divergence)  |  Code (negative stepsize)
    Inspired by a recent result that negative stepsizes make gradient descent ascent converge on bilinear counterexamples, this post shows that periodically using a negative dual stepsize also fixes the divergence of multi-block ADMM on classical quadratic counterexamples.

AI-assisted results

The following results were developed with AI assistance with accompanying Lean projects.

  1. An $\Omega(T^{-3/2})$ lower bound for heavy ball.
    GPT-6 Astra, prompted by Wenzhi Gao. 09/09/2026.
    PDF  |  LEAN project (.zip)
    This result proposes a worst-case last-iterate lower bound of $\Omega(LR^2/T^{3/2})$ for every predetermined heavy-ball schedule with nonnegative stepsizes and momenta in $[0,1)$, using a static Huber chain construction.
  2. An $\Omega(\sqrt{n})$ lower bound for infeasible-start long-step interior-point methods.
    GPT-6 Astra, prompted by Wenzhi Gao and Huikang Liu. 09/05/2026.
    PDF  |  LEAN project (.zip)
    This result proposes an explicit reverse-Helmert linear program yielding an $\Omega(\min\{n,\sqrt{n}\log(1/\varepsilon)\})$ lower bound on Newton-direction calls for the specified class of scalar-centered infeasible-start long-step methods.