BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Neural Attention - Elre Oldewage\, George Hron
DTSTART:20191120T140000Z
DTEND:20191120T153000Z
UID:TALK135190@talks.cam.ac.uk
CONTACT:Robert Pinsler
DESCRIPTION:Sequence-to-sequence models have been very successful in natur
 al language processing tasks such as language translation. Self-attention 
 - an attention mechanism that relates different positions of a single sequ
 ence - has proved to be an important technique that significantly improves
  the quality of model outputs. We will discuss attention in the context of
  neural machine translation and introduce the transformer model. Transform
 ers rely entirely on self-attention to capture long-range dependencies\, r
 ather than recurrence or convolutions. By dispensing with recurrence\, tra
 nsformers are also more parallelizable than recurrent models\, which are i
 nherently sequential. We consider a number of case studies\, most notably 
 BERT\, which is the major success story for self-attention. We also consid
 er applications of attention to images and graph neural networks.
LOCATION:Engineering Department\, CBL Room BE-438
END:VEVENT
END:VCALENDAR