Abstract

Contemporary Reading Comprehension (RC) datasets — SQuAD, TriviaQA, etc. — are dominated by queries that can be answered with a single paragraph or document. However, enabling models to combine pieces of textual information from different sources would drastically extend the scope of RC. In this talk, I will introduce a novel Multi-hop RC task, where a model has to learn how to find and combine disjoint pieces of textual evidence, effectively performing multi-step (alias multi-hop) inference. I present two datasets, WikiHop and MedHop, from different domains — both constructed using a unified methodology. I will then discuss the behaviour of several baseline models, including two established end-to-end RC models, BiDAF and FastQA. For example, one model is in fact capable of integrating information across documents, but both models struggle to select relevant information. Overall the end-to-end models outperform multiple baselines, but their best accuracy is still far behind human performance, leaving ample room for model improvement. It is our hope that these new datasets will drive future RC model development, leading to new and improved applications in areas such as Search, Question Answering, and Fact Checking. Paper: https://arxiv.org/abs/1710.06481