BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:NMT Analysis: The Trade-Off Between Source and Target\, and (a Bit
  of) the Training Process - Elena Voita (University of Edinburgh)
DTSTART:20210618T110000Z
DTEND:20210618T120000Z
UID:TALK160507@talks.cam.ac.uk
CONTACT:Huiyuan Xie
DESCRIPTION:Join Zoom Meeting \nhttps://cl-cam-ac-uk.zoom.us/j/91424580226
 ?pwd=WHFKRW1ORCtBck15SUVOdXowd29uUT09 \n\nMeeting ID: 914 2458 0226 \nPass
 code: 333459\n\nIn Neural Machine Translation (and\, more generally\, cond
 itional language modeling)\, the generation of a target token is influence
 d by two types of context: the source and the prefix of the target sequenc
 e. While many attempts to understand the internal workings of NMT models h
 ave been made\, none of them explicitly evaluates relative source and targ
 et contributions to a generation decision. We propose a way to explicitly 
 evaluate these relative source and target contributions to the generation 
 process\, and analyse NMT Transformer. When looking at changes in the cont
 ributions when conditioning on different types of prefixes\, we show that 
 models suffering from exposure bias are more prone to over-relying on targ
 et history (and hence to hallucinating) than the ones where the exposure b
 ias is mitigated. Additionally\, we analyze changes in the source and targ
 et contributions when varying the amount of training data\, and during the
  training process. We find that models trained with more data tend to rely
  on source information more and to have more sharp token contributions\; t
 he training process is non-monotonic with several stages of different natu
 re. If we have time\, I’ll also talk about our ongoing work that takes a
  closer look at the phenomena learned during these training stages.
LOCATION:Virtual (Zoom)
END:VEVENT
END:VCALENDAR