BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Zero-shot learning and out-of-distribution generalization: two sid
 es of the same coin - Jonathan Berant\, Tel Aviv University
DTSTART:20211202T110000Z
DTEND:20211202T120000Z
UID:TALK166576@talks.cam.ac.uk
CONTACT:Haim Dubossarsky
DESCRIPTION:Recent advances in large pre-trained language models have shif
 ted the NLP community’s attention to new challenges: (a) training models
  with zero\, or very few\, examples\, and (b) generalizing to out-of-distr
 ibution examples. In this talk\, I will argue that the two are intimately 
 related\, and describe ongoing (read\, new!) work in those directions. Fir
 st\, I will describe a new pre-training scheme for open-domain question an
 swering that is based on the notion of “recurring spans” across differ
 ent paragraphs. We show this training scheme leads to a zero-shot retrieve
 r that is competitive with DPR (which trains on thousands of examples)\, a
 nd is more robust w.r.t the test distribution. Second\, I will focus on co
 mpositional generalization\, a particular type of out-of-distribution gene
 ralization setup where models need to generalize to structures that are un
 observed at training time. I will show that the view that seq2seq models c
 ategorically do not generalize to new compositions is false\, and present 
 a more nuanced analysis\, which elucidates what are the conditions under w
 hich models struggle to compositionally generalize.
LOCATION:https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBd
 XVpOXFvdz09
END:VEVENT
END:VCALENDAR