BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Retrieval Augmented NLP - Jiri Hron and Nitarshan Rajkumar (Univer
 sity of Cambridge)
DTSTART:20220223T110000Z
DTEND:20220223T123000Z
UID:TALK169043@talks.cam.ac.uk
CONTACT:Elre Oldewage
DESCRIPTION:State-of-the-art language models (LMs) are now so large that o
 nly a handful of labs can afford to train them. The scale is driven by the
  training setup which forces the LM to memorise as much factual informatio
 n as possible. However\, humans already have a tool for finding informatio
 n: search engines. We will discuss the recent trend of extending LMs with 
 an information retrieval component\, which allows order of magnitude small
 er models to outcompete even the largest LMs. We will specifically focus o
 n the most recent developments: LaMDA\, WebGPT\, and RETRO. Finally\, we w
 ill touch on the design of modern search engines\, and draw parallels to r
 etrieval augmented LMs.\n\nRecommended reading:\n\n# RETRO: https://arxiv.
 org/abs/2112.04426\n# WebGPT: https://arxiv.org/abs/2112.09332\n# LaMDA: h
 ttps://arxiv.org/abs/2201.08239\n\nOur reading groups are live-streamed vi
 a Zoom and recorded for our Youtube channel. The Zoom details are distribu
 ted via our weekly mailing list.
LOCATION: Cambridge University Engineering Department \,LR3A
END:VEVENT
END:VCALENDAR