Question Answering (QA) is an increasingly important topic in NLP with the proliferation of chatbots and virtual assistants. In this project a QA system is built by exploring two end-to-end models: Firstly, the baseline BiDAF model was improved by adding a character embedding layer with multiple convolutional layers, an extra embeddings attention layer which captures the "summary" of the embedding vectors, a context-to-context self-attention layer, gated recurrent units (GRU) and Swish activation. Secondly, the QANet model was re-implemented from scratch and successfully explored some hyperparameter finetunings to improve performance.
The improved BiDAF model (SA-BiDAF++) incorporating self-attention, achieved 65.3 EM / 68.8 F1 scores on the test set of the SQuAD 2.0. That is a clear indication that architecture fine-tunings and optimizations can improve significantly the performance of non-PCE models.
