University of Cambridge > Talks.cam > Language Technology Lab Seminars > Adapters in Transformers. A New Paradigm for Transfer Learning…?

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹ️ Information

Adapters in Transformers. A New Paradigm for Transfer Learning…?

Download to your calendar using vCal

If you have a question about this talk, please contact Marinela Parovic .

Adapters have recently been introduced as an alternative transfer learning strategy. Instead of fine-tuning all weights of a pre-trained transformer-based model, small neural network components are introduced at every layer. While the pre-trained parameters are frozen, only the newly introduced adapter weights are fine-tuned, achieving an encapsulation of the down-stream task information in designated parts of the model. In this talk we will provide an introduction to adapter-training in natural language processing. We will go into detail on how the encapsulated knowledge can be leveraged for compositional transfer learning, as well as cross-lingual transfer. We will briefly touch the efficiency of adapters in terms of trainable parameters as well as (wall-clock) training time. Finally, we will provide an outlook to recent alternative adapter approaches and training strategies.

This talk is part of the Language Technology Lab Seminars series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

McCrum Lecture Theatre, Bene't Street Climate Change and Sustainability in Multiple Dimensions Talks related to sustainability and the environment

Other talks

Morning Break The LMC-SMC System: A New Laboratory for Galaxy Evolution, Dynamics and Dark Matter Content, Caching and Kubernetes: Performance and User Experience in the Browser Title TBC Bigger Picture Talk: Developing future marine engines running on ammonia and other green fuels Caribbean Trip and Cacti
 

© 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity