BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Modular and Compositional Transfer Learning - Jonas Pfeiffer (Goog
 le Research)
DTSTART:20230224T120000Z
DTEND:20230224T130000Z
UID:TALK196519@talks.cam.ac.uk
CONTACT:Rami Aly
DESCRIPTION:Abstract:\n\nWith pre-trained transformer-based models continu
 ously increasing in size\, there is a dire need for parameter-efficient an
 d modular transfer learning strategies. In this talk\, we will touch base 
 on adapter-based fine-tuning\, where instead of fine-tuning all weights of
  a model\, small neural network components are introduced at every layer. 
 While the pre-trained parameters are frozen\, only the newly introduced ad
 apter weights are fine-tuned\, achieving an encapsulation of the down-stre
 am task information in designated parts of the model. We will demonstrate 
 that adapters are modular components which can be composed for improvement
 s on a target task and how they can be used for out of distribution genera
 lization on the example of zero-shot cross-lingual transfer. Finally\, we 
 will discuss how adding modularity during pre-training can mitigate catast
 rophic interference and consequently lift the curse of multilinguality.\n\
 nBio:\n\nJonas Pfeiffer is a Research Scientist at Google Research. He is 
 interested in modular representation learning in multi-task\, multilingual
 \, and multi-modal contexts\, and in low-resource scenarios. He worked on 
 his PhD at the Technical University of Darmstadt\,  was a visiting researc
 her at the New York University and a Research Scientist Intern at Meta Res
 earch. Jonas has received the IBM PhD Research Fellowship award for 2021/2
 022. He has given numerous invited talks at academia\, industry and ML sum
 mer schools\, and has co-organized multiple workshops on multilinguality a
 nd multimodality.\n\nTopic: NLIP Seminar\nTime: Feb 24\, 2023 12:00 PM Lon
 don\n\nJoin Zoom Meeting\nhttps://cl-cam-ac-uk.zoom.us/j/94330375053?pwd=T
 jRtbTg5aUdzWVdLRU15RjR0V2g0Zz09\n\nMeeting ID: 943 3037 5053\nPasscode: 76
 8471
LOCATION:Virtual (Zoom)
END:VEVENT
END:VCALENDAR