University of Cambridge > Talks.cam > Machine Learning Reading Group @ CUED > Retrieval Augmented NLP

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Retrieval Augmented NLP

Download to your calendar using vCal

If you have a question about this talk, please contact Elre Oldewage .

State-of-the-art language models (LMs) are now so large that only a handful of labs can afford to train them. The scale is driven by the training setup which forces the LM to memorise as much factual information as possible. However, humans already have a tool for finding information: search engines. We will discuss the recent trend of extending LMs with an information retrieval component, which allows order of magnitude smaller models to outcompete even the largest LMs. We will specifically focus on the most recent developments: LaMDA, WebGPT, and RETRO . Finally, we will touch on the design of modern search engines, and draw parallels to retrieval augmented LMs.

Recommended reading:

  1. RETRO : https://arxiv.org/abs/2112.04426
  2. WebGPT: https://arxiv.org/abs/2112.09332
  3. LaMDA: https://arxiv.org/abs/2201.08239

Our reading groups are live-streamed via Zoom and recorded for our Youtube channel. The Zoom details are distributed via our weekly mailing list.

This talk is part of the Machine Learning Reading Group @ CUED series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Pharmacology Lecture Theatre. Newnham College Acoustics Lab, CUED

Other talks

Group Work Title TBC Toward affordable preventive healthcare CPH IRC and PHI Dinner Alicia Anderson discussing some exciting news about HARPS3
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity