BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Fundamental limits of generative AI - Helmut Bölcskei - ETH Zuric
 h
DTSTART:20230428T130000Z
DTEND:20230428T140000Z
UID:TALK189104@talks.cam.ac.uk
CONTACT:Randolf Altmeyer
DESCRIPTION:Generative AI has seen tremendous successes recently\, most no
 tably the chatbot ChatGPT and the DALLE2 software creating realistic image
 s and artwork from text descriptions. Underlying these and other generativ
 e AI systems are usually neural networks trained to produce text\, images\
 , audio\, or video from text inputs. The aim of this talk is to develop an
  understanding of the fundamental capabilities of generative neural networ
 ks. Specifically and mathematically speaking\, we consider the realization
  of high-dimensional random vectors from one-dimensional random variables 
 through deep neural networks. The resulting random vectors follow prescrib
 ed conditional probability distributions\, where the conditioning represen
 ts the text input of the generative system and its output can be text\, im
 ages\, audio\, or video. It is shown that every d-dimensional probability 
 distribution can be generated through deep ReLU networks out of a 1-dimens
 ional uniform input distribution. What is more\, this is possible without 
 incurring a cost—in terms of approximation error as measured in Wasserst
 ein-distance—relative to generating the d-dimensional target distributio
 n from d independent random variables. This is enabled by a space-filling 
 approach which realizes a Wasserstein-optimal transport map and elicits th
 e importance of network depth in driving the Wasserstein distance between 
 the target distribution and its neural network approximation to zero. Fina
 lly\, we show that the number of bits needed to encode the corresponding g
 enerative networks equals the fundamental limit for encoding probability d
 istributions (by any method) as dictated by quantization theory according 
 to Graf and Luschgy. This result also characterizes the minimum amount of 
 information that needs to be extracted from training data so as to be able
  to generate a desired output at a prescribed accuracy and establishes tha
 t generative ReLU networks can attain this minimum.\n\nThis is joint work 
 with D. Perekrestenko and L. Eberhard
LOCATION:  Centre for Mathematical Sciences MR12\, CMS
END:VEVENT
END:VCALENDAR