![[Search]](/static/images/search.gif){kind=small}
![[A-Z Index]](/static/images/az.gif){kind=small}
![[Contact]](/static/images/contact.gif){kind=small}
![[University of Cambridge]](/static/images/identifier2.gif){kind=small}
![[Talks.cam]](/static/images/talkslogosmall.gif)
Petar Veliฤkoviฤ (Google Deepmind) 
Thursday 10 October 2024, 14:00-15:00
The Transformer (OOD) House of Cards
- ๐ค Speaker: Petar Veliฤkoviฤ (Google Deepmind) ๐ Website
- ๐ Date & Time: Thursday 10 October 2024, 14:00 - 15:00
- ๐ Venue: Maxwell Centre
Abstract
The Transformer architecture has certainly been the landmark deep learning model in recent years, enabling seamless integration of information across many different modalities and surprisingly insightful behaviours emerging at scale. However, in spite of the very challenging problems that are now within reach of Transformers, they are also seemingly unable to robustly perform when faced with variations of, comparatively, much simpler problems. We attribute this to shaky foundations: there are certain kinds of computations that are always going to be out of reach of Transformers, no matter how well we train them—and a lot of such computations occur outside of the distribution the model was trained on. In this talk, I will outline some of these cracks in the system we’ve discovered, as well as ideas for the way forward towards building generally intelligent agents of the future.
Series This talk is part of the Data Intensive Science Seminar Series series.
Included in Lists
- bld31
- Cambridge Astronomy Talks
- Cambridge Centre for Data-Driven Discovery (C2D3)
- Cambridge talks
- Chris Davis' list
- Combined External Astrophysics Talks DAMTP
- Cosmology, Astrophysics and General Relativity
- Institute of Astronomy Extra Talks
- Institute of Astronomy Talk Lists
- Interested Talks
- Maxwell Centre
- ndk22's list
- ob366-ai4er
- rp587
- Titel: TBC
- Trust & Technology Initiative - interesting events
- yk449
Note: Ex-directory lists are not shown.