BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Scalable Approaches to Self-Supervised Learning using Spectral Ana
 lysis - Ross Viljoen and Vincent Dutordoir (University of Cambridge)
DTSTART:20230405T100000Z
DTEND:20230405T113000Z
UID:TALK199249@talks.cam.ac.uk
CONTACT:James Allingham
DESCRIPTION:Learning the principal eigenfunctions of an operator is a fund
 amental problem in various machine learning tasks\, from representation le
 arning to Gaussian processes. However\, traditional non-parametric solutio
 ns suffer from scalability issues -- rendering them impractical on large d
 atasets.\n\nThis reading group will discuss parametric approaches to appro
 ximating eigendecompositions using neural networks. In particular\, Spectr
 al Inference Networks (SpIN) offer a scalable method for approximating eig
 enfunctions of symmetric operators on high-dimensional function spaces usi
 ng bi-level optimization methods and gradient masking (Pfau et al.\, 2019)
 .\n\nA recent improvement on SpIN\, called NeuralEF\, focuses on approxima
 ting eigenfunction expansions of kernels (Deng et al.\, 2022a). The method
  is applied to modern neural-network based kernels (GP-NN and NTK) as well
  as scaling up the linearised Laplace approximation for deep networks (Den
 g et al.\, 2022a). Finally\, self-supervised learning can be expressed in 
 terms of approximating a contrastive kernel\, which allows NeuralEF to lea
 rn structured representations (Deng et al.\, 2022b).\n\nReferences\n\nDavi
 d Pfau\, Stig Petersen\, Ashish Agarwal\, David G. T. Barrett\,and Kimberl
 y L. Stachenfeld. "Spectral inference networks: Unifying deep and spectral
  learning." ICLR (2019).\n\nZhijie Deng\, Jiaxin Shi\, and Jun Zhu. "Neura
 lEF: Deconstructing kernels by deep neural networks." ICML (2022a).\n\nZhi
 jie Deng\, Jiaxin Shi\, Hao Zhang\, Peng Cui\, Cewu Lu\, Jun Zhu. "Neural 
 Eigenfunctions Are Structured Representation Learners." arXiv preprint arX
 iv:2210.12637 (2022b).
LOCATION:Cambridge University Engineering Department\, CBL Seminar room BE
 4-38.
END:VEVENT
END:VCALENDAR