BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Paying Attention to Efficiency: LLM Deployment on Mobile and Edge 
 Devices - Stefanos Laskaridis\, Brave Software
DTSTART:20241105T130000Z
DTEND:20241105T140000Z
UID:TALK223354@talks.cam.ac.uk
CONTACT:Mateja Jamnik
DESCRIPTION:Transformers have recently sparked significant interest in AI\
 , driving advancements in accuracy and enabling a wide range of applicatio
 ns\, from multi-modal intelligent assistants to autonomous systems. While 
 their scaling laws promise even greater capabilities\, the demands on hard
 ware and data present significant challenges. In response\, there is growi
 ng interest in compressing these models to smaller\, more efficient forms\
 , making them feasible for deployment with lower resource requirements. As
  edge and mobile devices are integrating increasingly powerful System-On-C
 hips (SoCs)\, deploying these models locally becomes viable\, thus enablin
 g new use-cases while enhancing privacy\, sustainability and task-specific
  customization.\n\nIn this talk\, I will be touching upon two areas: first
 \, measuring the execution efficiency and deployability of Large Language 
 Models (LLMs) on mobile and edge devices\; and second optimising DNN workl
 oads for efficiency through low-rank decompositions. I will introduce  MEL
 T (MobiCom'24)\, a benchmarking framework designed to assess the computati
 onal\, memory\, energy\, and thermal characteristics of LLMs running on de
 vice\, identifying associated bottlenecks. Following this\, I will present
  Maestro (ICML'24)\, a novel approach leveraging trainable low-rank decomp
 ositions to enable more efficient training and deployment of DNNs\, enable
 d via data-informed progressive shrinking of networks.\n\n"You can also jo
 in us on Zoom":https://cam-ac-uk.zoom.us/j/83400335522?pwd=LkjYvMOvVpMbabO
 V1MVTm8QU6DrGN7.1\n\n
LOCATION:Lecture Theatre 2\, Computer Laboratory\, William Gates Building
END:VEVENT
END:VCALENDAR