BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:A Tutorial on Algorithmic Information Theory in Modern ML - Szilvi
 a Ujvary\, Arik Reuter\, Xianda Sun (University of Cambridge)
DTSTART:20251105T110000Z
DTEND:20251105T123000Z
UID:TALK240613@talks.cam.ac.uk
CONTACT:Xianda Sun
DESCRIPTION:This tutorial explores how ideas from algorithmic information 
 theory connect to modern machine learning through three recent papers. We 
 begin with Solomonoff induction—the theoretically optimal but uncomputab
 le predictor—and show how neural networks can approximate it by training
  on Universal Turing Machine data (Grau-Moya et al.\, 2024). We then estab
 lish the formal foundations by examining Kolmogorov complexity and its con
 nections to compression and randomness in images\, exploring how the Solom
 onoff prior helps us understand what makes images "realistic" and guides t
 he design of better generative models and anomaly detectors (Theis\, 2024)
 . Finally\, we demonstrate these principles at scale\, deriving non-vacuou
 s generalization bounds for large language models with billions of paramet
 ers through compression-based analysis using the SubLoRA technique (Lotfi 
 et al.\, 2024). No prior background in algorithmic information theory requ
 ired—we'll build intuition from first principles while connecting to fam
 iliar ML concepts throughout.\n\nPapers:\n# Learning Universal Predictors 
 (Grau-Moya et al.\, 2024) - https://arxiv.org/abs/2401.14953\n# What Makes
  an Image Realistic? (Theis\, 2024) - https://arxiv.org/abs/2403.04493\n# 
 Non-Vacuous Generalization Bounds for Large Language Models (Lotfi et al.\
 , 2024) - https://arxiv.org/abs/2312.17173
LOCATION:Cambridge University Engineering Department\, CBL Seminar room BE
 4-38.
END:VEVENT
END:VCALENDAR