BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Behavioral machine learning - Dr. Keyon Vafa\, Harvard Data Scienc
 e Initiative
DTSTART:20250206T160000Z
DTEND:20250206T170000Z
UID:TALK226252@talks.cam.ac.uk
CONTACT:Shun Shao
DESCRIPTION:Abstract:\n\nWhile benchmark datasets have been important for 
 making progress in supervised learning\, they cannot capture the diversity
  of uses afforded by large language models and other general-purpose techn
 ologies. How can we trust metrics that don't capture our experience using 
 these models? This talk presents alternative approaches to evaluation that
  incorporate behavioral models of how people use them. In the first part o
 f the talk\, we study a human generalization function that arises when peo
 ple make inferences about what an LLM can do based on their interactions w
 ith it. This motivates a new form of alignment: the best LLM is the one th
 at allows people to make the most accurate inferences about where it will 
 or won't succeed. In the second part of the talk\, we describe a new commo
 n task for evaluating how well people can steer generative models toward d
 esired outputs. We find that humans struggle to steer text-to-image models
 \, and we propose a new method\, reinforcement learning for human steering
  (RLHS)\, that empirically improves steerability. \n\nBio:\n\nKeyon Vafa i
 s a postdoctoral fellow at the Harvard Data Science Initiative. His resear
 ch focuses on developing ML methods to address economic questions along wi
 th using insights from the behavioral sciences to improve ML methods. Keyo
 n completed his PhD in computer science from Columbia University\, where h
 e was an NSF GRFP Fellow and the recipient of the Morton B. Friedman Memor
 ial Prize for excellence in engineering. He was a co-organizer of the Neur
 IPS 2024 Workshop on Behavioral Machine Learning and is a member of the ea
 rly career board of the the Harvard Data Science Review. 
LOCATION:https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBd
 XVpOXFvdz09
END:VEVENT
END:VCALENDAR