BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Explanations as a Catalyst: Leveraging Large Language Models to Em
 brace Human Label Variation - Beiduo Chen
DTSTART:20251010T100000Z
DTEND:20251010T110000Z
UID:TALK237253@talks.cam.ac.uk
CONTACT:Shun Shao
DESCRIPTION:Abstract:\n\nHuman label variation (HLV)—the phenomenon wher
 e multiple annotators provide different yet valid labels for the same data
 —is a rich source of information often dismissed as noise. Capturing thi
 s variation is crucial for building robust NLP systems\, but doing so is t
 ypically resource-intensive. This talk presents a series of studies on how
  Large Language models (LLMs) can serve as a catalyst to embrace and model
  HLV\, moving from scalable approximation to a deeper analysis of the reas
 oning process itself.\n\nFirst\, I will discuss how LLMs can approximate f
 ull Human Judgment Distributions (HJDs) from just a few human-provided exp
 lanations. Our work shows that this explanation-based approach significant
 ly improves alignment with human judgments. This investigation also reveal
 s the limitations of traditional\, instance-level distribution metrics and
  highlights the importance of complementing them with global-level measure
 s to more effectively evaluate alignment.\n\nBuilding on this\, the second
  part of the talk addresses the high cost of collecting human explanations
  by asking: can LLM-generated explanations serve as a viable proxy? We dem
 onstrate that when guided by a few human labels\, explanations generated b
 y LLMs are indeed effective proxies\, achieving comparable performance to 
 human-written ones in approximating HJDs. This finding opens up a scalable
  and efficient pathway for modeling HLV\, especially for datasets where hu
 man explanations are not available.\n\nFinally\, I will shift from post-ho
 c explanation (justifying a given answer) to a forward-reasoning paradigm.
  I will introduce CoT2EL\, a novel pipeline that extracts explanation-labe
 l pairs directly from an LLM's Chain-of-Thought (CoT) process before a fin
 al answer is selected. This method allows us to analyze the model's reason
 ing across multiple plausible options. To better assess these nuanced judg
 ments\, I will also present a new rank-based evaluation framework that pri
 oritizes the ordering of answers over exact distributional scores\, showin
 g a stronger alignment with human decision-making.\n\nBio\n\nBeiduo Chen i
 s a PhD student at the MaiNLP lab at LMU Munich\, supervised by Prof. Barb
 ara Plank. He is also a member of the European Laboratory for Learning and
  Intelligent Systems (ELLIS) PhD Program\, co-supervised by Prof. Anna Kor
 honen at University of Cambridge. He received his Master's and Bachelor's 
 degrees from the University of Science and Technology of China. His resear
 ch focuses on human-centered NLP\, with a special emphasis on the uncertai
 nty\, trustworthiness\, and evaluation of Large Language Models. He has pu
 blished several papers in top-tier NLP conferences\, including ACL and EMN
 LP.
LOCATION:GR03\, English Faculty Building\, 9 West Road\, Sidgwick Site and
  online https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBdX
 VpOXFvdz09
END:VEVENT
END:VCALENDAR