BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Measuring Political Bias in Large Language Models - Paul Röttger 
 (Bocconi University)
DTSTART:20250516T110000Z
DTEND:20250516T120000Z
UID:TALK230203@talks.cam.ac.uk
CONTACT:Suchir Salhan
DESCRIPTION:Large language models (LLMs) are helping millions of users to 
 learn and write about a diversity of issues. In doing so\, LLMs may expose
  users to new ideas and perspectives\, or reinforce existing knowledge and
  user opinions. This creates concerns about political bias in LLMs\, and h
 ow these biases might influence LLM users and society. In my talk\, I will
  first discuss why measuring political biases in LLMs is difficult\, and w
 hy most evidence so far should be approached with skepticism. Using the Po
 litical Compass Test as a case study\, I will demonstrate critical issues 
 of robustness and ecological validity when applying such tests to LLMs. Se
 cond\, I will present our approach to building a more meaningful evaluatio
 n dataset called IssueBench\, to measure biases in how LLMs write about po
 litical issues. I will describe the steps we took to make IssueBench reali
 stic and robust. Then\, I will outline our results from testing state-of-t
 he-art LLMs with IssueBench\, including clear evidence for issue bias\, st
 riking similarities in biases across models\, and strong alignment with De
 mocrat over Republican voter positions on a subset of issues.\nBio: Paul i
 s a postdoctoral researcher in the MilaNLP Lab at Bocconi University\, wor
 king on evaluating and improving the alignment and safety of large languag
 e models\, as well as measuring their societal impacts. For his recent wor
 k in this area\, he won Outstanding Paper at ACL and Best Paper at NeurIPS
  D&B. Before coming to Milan\, Paul completed his PhD at the University of
  Oxford\, where he worked on LLMs for hate speech detection. During his Ph
 D\, Paul also co-founded Rewire\, a start-up building AI for content moder
 ation\, which was acquired by another large online safety company in 2023.
 \n
LOCATION:Room FW26 with Hybrid Format. Here is the Zoom link for those tha
 t wish to join online: https://cam-ac-uk.zoom.us/j/4751389294?pwd=Z2ZOSDk0
 eG1wZldVWG1GVVhrTzFIZz09
END:VEVENT
END:VCALENDAR