Most of my work comes back to one question: when can a system act on the signal it is given? In my PhD I trained multi-agent reinforcement learning policies to schedule large stochastic production networks, and at Samsung Display I shipped one — a scheduler for lines with hundreds of machines, where a reward or a constraint that is slightly wrong costs real money. Six of my patent applications are about exactly this: reweighting rewards, adapting a policy to a new reward, and letting a safety policy step in.
With LLM agents the question comes back in a harder form. Where a simulator or a design-rule checker can score the outcome, as in the chip-design environments my team builds, the reward can be trusted. Most agent tasks can only be partly checked, and the usual answer is to ask another model. In my own measurements, the LLM judges that agreed most with human labels still accepted plans that broke the user's budget — a shortcut an RL policy could learn to exploit. So I build verifiers that pass only what code can confirm (arithmetic, dates, quoted text) and say “cannot verify” otherwise.
The other thing I have learned is that a number is only as good as the way it was measured. I freeze held-out sets, write down what counts as a result before the run, measure once, and report the runs that did not work.