BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Estimating and Calibrating Uncertainty in LLMs - Desi Ivanova (Uni
 versity of Oxford)
DTSTART:20250828T130000Z
DTEND:20250828T133000Z
UID:TALK234517@talks.cam.ac.uk
DESCRIPTION:Large Language Models (LLMs) often sound certain when they are
  wrong. With rapid adoption in critical sectors like healthcare\, scientif
 ic discovery and research automation\, where the consequences of errors ca
 n be substantial\, reliable Uncertainty Quantification (UQ) becomes crucia
 l for safe and trustworthy deployment. In this talk\, we will first review
  the current landscape of UQ methods for LLMs and discuss the trade-offs b
 etween calibration\, discrimination\, and compute. We'll then look at how 
 generation temperature (the scalar that controls generative diversity) aff
 ects calibration at a semantic level. We'll introduce a semantic calibrati
 on framework and show that simple post-hoc temperature scaling markedly im
 proves meaning-level calibration and selective prediction across open- and
  closed-book question-answering tasks.
LOCATION:Seminar Room 1\, Newton Institute
END:VEVENT
END:VCALENDAR