In this work, we build a question answering (QA) system and apply it on the Stanford Question Answering Dataset, version 2.0. Our goal is to achieve strong performance on this task without using pre-trained language models. Our primary contribution is a highly performant implementation of the QANet model. Additionally, we experiment with various modifications to this architecture. Most notably, we show that modifying the output layer, such that answer span's ending position prediction is a function of the starting position prediction, yields significant improvements over the original design. Using a QANet ensemble, we reach an F1 score of 71.87 and an EM score of 68.89 on an unseen test set (rank #1 out of 100+ submissions to the test leaderboard for the IID SQuAD Track of CS 224N at Stanford, Winter 2021).
