Vision-Language Models & Multimodal Learning Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Vision-language models learn a shared space for images and text, enabling zero-shot transfer and instruction following. This reading list moves from early contrastive and fusion encoders like ALIGN, ViLT and UNITER to modern instruction-tuned systems such as BLIP-2, LLaVA and Kosmos-2.

Vision-Language Models & Multimodal Learning: 10 key papers

1. Learning Transferable Visual Models From Natural Language Supervision
   Radford et al. arXiv 2021.
2. Flamingo: a Visual Language Model for Few-Shot Learning
   Alayrac et al. arXiv 2022.
3. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
   Li et al. arXiv 2023.
4. Visual Instruction Tuning
   Liu et al. arXiv 2023.
5. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
   Jia et al. arXiv 2021.
6. Zero-Shot Text-to-Image Generation
   Ramesh et al. arXiv 2021.
7. Sigmoid Loss for Language Image Pre-Training
   Zhai et al. arXiv 2023.
8. PaLM-E: An Embodied Multimodal Language Model
   Driess et al. arXiv 2023.
9. Segment Anything
   Kirillov et al. arXiv 2023.
10. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
    Dosovitskiy et al. arXiv 2020.

Related reading lists

• Instruction Following & Finetuning
• Contrastive & Self-Supervised Learning
• Data Augmentation for Reinforcement Learning
• Prompt Optimization & In-Context Learning for LLMs
• Retrieval-Augmented Generation
• Scaling Laws and Emergent Capabilities
• Reasoning in LLMs
• Knowledge Distillation