Answer Pointer Inspired BiDAF And QANet For Machine Comprehension

Imagine that you are trying to find the answer for a question given a context paragraph. This kind of tasks fall into the category of one of the hottest topics in NLP - machine comprehension. With the help of emerging high-performance GPUs, deep learning for machine comprehension has progressed tremendously. RNN based methods, such as Match-LSTM and Bidirectional Attention Flow (BiDAF), and transformer-like methods, such as QANet, keep pushing the performance boundary of machine comprehension on the SQuAD datasets. Our team proposes to improve the performance of the baseline BiDAF and the QANet models on SQuAD 2.0. We replace the original output layer of BiDAF and QANet with Answer Pointer inspired output layers and add character level embedding and ReLU MLP fusion function to the baseline BiDAF model. We achieve significantly better performance using ensemble learning with majority voting on modified BiDAF, QANet1, and QANet3 models. Specifically, the ensemble learning achieves a F1 score of 66.219 and a EM score of 62.840 on the test datasets and a F1 score of 68.024 and a EM score of 64.561 on the validation datasets.