Context Demonstrations and Backtranslation Augmentation Techniques For a More Robust QA System

Because many real-world NLP tasks rely on user data that is not necessarily guaranteed to be in-distribution, it is critical to build robust question answering systems that can generalize to out-of-domain data. We aim to build a question answering system using context demonstrations and dataset augmentation via backtranslation on top of DistilBERT that is robust to domain shifts. Our method replicates one of the two approaches described in Gao et al. (2020), sampling and appending out-of-domain demonstrations to each training example when finetuning the model. Our method also augments the out-of-domain dataset from which demonstrations are sampled using backtranslation to generate in-distribution training examples. We find that the basic approach of simply appending randomly sampled out-of-domain demonstrations to in-domain contexts does not improve model F1 and EM score performance, but supplementing this approach by adding separator tokens between each demonstration and augmenting the out-of-domain training dataset using backtranslation improves model performance.