Abstract

Textual data exhibit pronounced variation along geographical dimensions (e.g., due to dialect differences). Common practices of training and deploying language models do not take this inherent dynamicity into account, leading to detrimental effects for their robustness and performance on downstream tasks. In this talk, I will explore how some shortcomings of text-only NLP pipelines can be alleviated by grounding language models in geography. I will first give a brief overview of prior research on geographical variation in NLP . I will then present geoadaptation, a method for geographically grounding language models that combines language modeling with geolocation prediction in a multi-task learning setup. Geoadaptation leads to consistent performance improvements across a range of tasks and language areas, especially in zero-shot settings. Finally, I will show that the effectiveness of geoadaptation stems from its ability to geographically retrofit the representation space of language models.