In-Context RL & Prompting Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

In-context RL treats learning itself as sequence prediction: a transformer pretrained on histories can improve by prompting, not gradient updates. This list covers algorithm distillation, decision-pretrained transformers, and prompting for RL.

In-Context RL & Prompting: 10 key papers

1. Supervised Pretraining Can Learn In-Context Reinforcement Learning
   Lee et al. arXiv 2023.
2. Emergence of In-Context Reinforcement Learning from Noise Distillation
   Zisman et al. arXiv 2023.
3. In-context Exploration-Exploitation for Reinforcement Learning
   Dai et al. arXiv 2024.
4. In-context Reinforcement Learning with Algorithm Distillation
   Laskin et al. arXiv 2022.
5. Transformers learn in-context by gradient descent
   von Oswald et al. arXiv 2022.
6. What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
   Garg et al. arXiv 2022.
7. An Explanation of In-context Learning as Implicit Bayesian Inference
   Xie et al. arXiv 2021.
8. Structured State Space Models for In-Context Reinforcement Learning
   Lu et al. arXiv 2023.
9. AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents
   Grigsby et al. arXiv 2023.
10. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
    Min et al. arXiv 2022.

Related reading lists

• Prompt Optimization & In-Context Learning for LLMs
• Transformers for Sequential Decision Making
• Chain-of-Thought & Reasoning
• State Space Models & Structured SSMs
• Efficient Transformers
• Knowledge Distillation
• Multi-Task & Transfer Reinforcement Learning
• Model Compression & Pruning