Our motivation is to build a Question Answering (QA) system that gives answers as specific and as accurate to queries, which is in itself an art but based on the science of Natural Language Processing (NLP). The main goal of our project is to produce a QA system that works well on SQuAD 2.0 dataset that performs better than the baseline Bidirectional Attention Flow (BiDAF) model. To better capture the context from a more expressive set of answers and understand the interactions between the question and the document, we utilized the coattention mechanism by encoding the two-way attention outputs together through a bidirectional reccurrent neural network (RNN). We experimented with enriching the embedding layer with concatenating character embeddings with existing word-level embedding, modifying the attention layer with coattention from Dynamic Coattention Networks (DCN), adding an Answer Pointer, which conditions the ending of the answer span on the starting position, to the output layer. Our best performing single model obtained F1/EM scores of 63.40/59.87, which both achieved better results than the baseline. Adding character embeddings and the answer pointer gave us a successful performance boost compared with the BiDAF baseline model. On the other hand, dynamic coattention from DCN did not beat the attention and modeling layer combined in the baseline BiDAF model but was worth trying. To further improve the performance of our model, we built ensemble models which finetune on the dropout rates, and the best one achieved F1/EM scores of 64.21/60.81.
