Memory-Augmented RL & Recurrent Policies Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Partial observability demands memory. This list covers recurrent policies, memory-augmented architectures, and transformers as memory.

Memory-Augmented RL & Recurrent Policies: 10 key papers

1. Neural Turing Machines
   Graves et al. arXiv 2014.
2. Hybrid computing using a neural network with dynamic external memory
   Graves et al. Nature 2016.
3. Model-Free Episodic Control
   Blundell et al. arXiv 2016.
4. Neural Episodic Control
   Pritzel et al. arXiv 2017.
5. Been There, Done That: Meta-Learning with Episodic Recall
   Ritter et al. arXiv 2018.
6. Unsupervised Predictive Memory in a Goal-Directed Agent
   Wayne et al. arXiv 2018.
7. Relational recurrent neural networks
   Santoro et al. arXiv 2018.
8. Deep Attention Recurrent Q-Network
   Sorokin et al. arXiv 2015.
9. Memory-based control with recurrent neural networks
   Heess et al. arXiv 2015.
10. Working Memory Graphs
    Loynd et al. arXiv 2019.

Related reading lists

• Long-Context & Memory Transformers
• Graph Neural Networks & Message Passing
• State Space Models & Structured SSMs
• Continual & Lifelong Learning
• Meta-Reinforcement Learning
• Few-Shot RL & Meta-Learning
• Neuroscience-Inspired Reinforcement Learning
• Transformers for Sequential Decision Making