BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Drawing Connections in Decentralized Deep Learning - Max Ryabinin
DTSTART:20250618T140000Z
DTEND:20250618T150000Z
UID:TALK233221@talks.cam.ac.uk
CONTACT:Sally Matthews
DESCRIPTION:Recently\, the field of Machine Learning has seen renewed inte
 rest in communication-efficient training over slow\, unreliable\, and hete
 rogeneous networks. While the latest results and their applications to LLM
 s are highly promising\, their underlying ideas have surprisingly many con
 nections to well-established approaches to distributed ML. In this talk\, 
 I will provide an overview of recent developments in decentralized trainin
 g within the broader context of areas such as volunteer computing\, commun
 ication-efficient optimization\, and federated learning. In addition\, I w
 ill present our research in this field\, ranging from Learning@home/DMoE t
 o Petals\, and share some lessons learned about ML research in general dur
 ing the development of these methods.\n\n\nMax Ryabinin is VP of Research 
 & Development at Together AI\, working on large-scale deep learning. Previ
 ously\, he was a Senior Research Scientist at Yandex\, studying a wide ran
 ge of topics in natural language processing and efficient machine learning
 . During his PhD\, he developed methods for distributed training and infer
 ence over slow and unstable networks\, such as DeDLOC\, SWARM Parallelism\
 , and Petals. He is also the creator and maintainer of Hivemind\, a highly
  popular open-source framework for decentralized training in PyTorch.
LOCATION:Computer Lab\, LT1
END:VEVENT
END:VCALENDAR