BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Long-Range Transformers - Valerii Likhosherstov\, University of Ca
 mbridge
DTSTART:20210303T110000Z
DTEND:20210303T123000Z
UID:TALK157297@talks.cam.ac.uk
CONTACT:Elre Oldewage
DESCRIPTION:These days\, Transformer architectures are showing state-of-th
 e-art performance in many tasks\, including natural language processing\, 
 computer vision\, protein modelling and beyond. Unfortunately\, Transforme
 rs scale quadratically (O(L^2)) as the sequence length L grows. In this ta
 lk\, we will discuss a zoo of recently proposed methods to reduce time or 
 memory complexity of Transformers up to O(L) and even O(1).\n \n\nLiteratu
 re:\n\nEfficient Transformers: A Survey. Yi Tay\, Mostafa Dehghani\, Dara 
 Bahri\, Donald Metzler. arXiv:2009.06732.\n \n\nRethinking Attention with 
 Performers. Krzysztof Choromanski\, Valerii Likhosherstov\, David Dohan\, 
 Xingyou Song\, Andreea Gane\, Tamas Sarlos\, Peter Hawkins\, Jared Davis\,
  Afroz Mohiuddin\, Lukasz Kaiser\, David Belanger\, Lucy Colwell\, Adrian 
 Weller. ICLR 2021.\n \n\nSub-Linear Memory: How to Make Performers SLiM. V
 alerii Likhosherstov\, Krzysztof Choromanski\, Jared Davis\, Xingyou Song\
 , Adrian Weller. arXiv:2012.11346.
LOCATION:https://eng-cam.zoom.us/j/86068703738?pwd=YnFleXFQOE1qR1h6Vmtwbno
 0LzFHdz09
END:VEVENT
END:VCALENDAR