BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Feedback Forensics: A Toolkit to Measure AI Personality - Arduin F
 indeis (University of Cambridge)
DTSTART:20250617T120000Z
DTEND:20250617T130000Z
UID:TALK232057@talks.cam.ac.uk
CONTACT:Mateja Jamnik
DESCRIPTION:Conventional AI benchmarks typically focus on the content of r
 esponses\, for example checking factual (e.g. MMLU) or mathematical correc
 tness (e.g. GSM8k). However\, for many language model applications\, the m
 anner (or "personality") of a model's responses also matters to users\, fo
 r example how friendly or confident responses are. Recent issues with mode
 l releases highlight the limited ability of existing evaluation approaches
  to capture such personality traits: a ChatGPT model version was rolled ba
 ck over sycophant personality issues\, other models' personalities have be
 en critised to overfit to the Chatbot Arena leaderboard.\n\nIn this talk\,
  I will introduce Feedback Forensics: our newly released toolkit to measur
 e AI personality traits. Using our toolkit\, I will first share results de
 tecting the personality traits currently encouraged by popular human feedb
 ack datasets (incl. Chatbot Arena). Next\, I will discuss changes and tren
 ds in personality traits exhibited across model families and versions. Fin
 ally\, I will take a closer look the personality differences between the C
 hatbot Arena and publicly released version of Llama-4-Maverick.\n\nThe tal
 k will feature a live demo of our personality visualisation tool and atten
 dees are invited to follow along via our online platform https://feedbackf
 orensics.com/ (laptops are encouraged).\n\n"You can also join us on Zoom":
 https://cam-ac-uk.zoom.us/j/83400335522?pwd=LkjYvMOvVpMbabOV1MVTm8QU6DrGN7
 .1\n
LOCATION:Lecture Theatre 2\, Computer Laboratory\, William Gates Building
END:VEVENT
END:VCALENDAR