Quantization for Efficient Inference Reading List

Curated by Mouhssine Rifaki | Stanford Electrical Engineering | Last updated August 2026

Key papers on quantizing neural networks for efficiency — from integer training to post-training LLM quantization with outliers and activation smoothing.

Quantization for Efficient Inference: 10 key papers

1. A White Paper on Neural Network Quantization
   Nagel et al. arXiv 2021.
2. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
   Jacob et al. arXiv 2017.
3. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
   Dettmers et al. arXiv 2022.
4. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
   Frantar et al. arXiv 2022.
5. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
   Lin et al. arXiv 2023.
6. QLoRA: Efficient Finetuning of Quantized LLMs
   Dettmers et al. arXiv 2023.
7. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
   Xiao et al. arXiv 2022.
8. Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
   Courbariaux et al. arXiv 2016.
9. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
   Rastegari et al. arXiv 2016.
10. Up or Down? Adaptive Rounding for Post-Training Quantization
    Nagel et al. arXiv 2020.

Related reading lists

• Model Compression & Pruning
• LLM Agents & Tool-Augmented Language Models
• Prompt Optimization & In-Context Learning for LLMs
• Reasoning in LLMs
• Certified Robustness
• Instruction Following & Finetuning
• Mechanistic Interpretability
• Active Learning & Adaptive Sampling