University of Cambridge > Talks.cam > Statistics > Provable representation learning in deep learning

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Provable representation learning in deep learning

Download to your calendar using vCal

If you have a question about this talk, please contact Dr Sergio Bacallado .

Deep representation learning seeks to learn a data representation that transfers to downstream tasks. In this talk, we study two forms of representation learning: supervised pre-training and self-supervised learning.

Supervised pre-training uses a large labeled source dataset to learn a representation, then trains a classifier on top of the representation. We prove that supervised pre-training can pool the data from all source tasks to learn a good representation which transfers to downstream tasks with few labeled examples.

Self-supervised learning creates auxiliary pretext tasks that do not require labeled data to learn representations. These pretext tasks are created solely using input features, such as predicting a missing image patch, recovering the colour channels of an image, or predicting missing words. Surprisingly, predicting this known information helps in learning a representation effective for downstream tasks. We prove that under a conditional independence assumption, self-supervised learning provably learns representations.

This talk is part of the Statistics series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

FW 11, William Gates Building. Zoom link: https://cl-cam-ac-uk.zoom.us/j/4361570789?pwd=Nkl2T3ZLaTZwRm05bzRTOUUxY3Q4QT09&from=addon Keynes Hall, Kings College Economic Epidemiology

Other talks

Title and abstract tba Discussion II Group Work Title TBC The organisational management of traumatic stress The spread of environmental cost-benefit analysis in UK government, c.1966-1995: representing the public in the Anthropocene
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity