An Analysis on the Effect of Domain Representations in Question Answering Models

Studies of robust reading comprehension models have included both learning domain specific representations and domain invariant representations. This project analyzes the effectiveness of each of these approaches using Mixture-of-Experts (MoE) and adversarial models. In the domain specific approach, MoE's form a single expert model for each input domain (Guo et al., 2018, Takahashi et al., 2019). In contrast, domain invariant models learn a generalized hidden representation that cannot distinguish the domain of the input (Ma et al., 2019, Lee et al., 2019). Additionally, models are assessed to determine their level of understanding of natural language against learning simple linguistic bias heuristics.