Recurrence, Transformers, and Beam Search - Oh My!

Question answering on the IID SQUAD 2.0 dataset is a proving ground for natural language processing systems. In this project, we explore recurrent and transformer-based architectures for SQuAD 2.0. We implement several improvements on the baseline BiDAF and the canonical transformer QANet. Our best model, BiDAF with character embeddings and beam search output, scores F1 62.291 and EM 59.493. Finally, we suggest further directions for research in self-attention and modeling/predicting NA answers.