BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Similarity-based Methods for Language Model Analysis and Predictio
 n - Julius Cheng (University of Cambridge)
DTSTART:20250318T130000Z
DTEND:20250318T140000Z
UID:TALK224350@talks.cam.ac.uk
CONTACT:Mateja Jamnik
DESCRIPTION:In natural language\, there are usually many ways to say the s
 ame thing: the answer to a question can be said multiple ways\, and there 
 are many good translations of the same sentence. As a result\, language mo
 dels (LMs) trained on large corpora often spread probability mass across a
  vast number of generations\, containing mostly minor variations. This rai
 ses problems for LM applications\; for prediction\, probability is loosely
  correlated with quality\, so various heuristics must be added to beam sea
 rch to achieve adequate results. For uncertainty quantification\, commonly
  used measures like Shannon entropy can overestimate uncertainty when prob
 ability is spread across functionally equivalent texts. In this talk\, I w
 ill present my PhD thesis work which addresses these shortcomings using me
 thods which incorporate measurements of semantic similarity. In prediction
 \, returning a "protoypical" prediction according to semantic similarity o
 utperforms high probability predictions. In uncertainty quantification\, g
 eneralizing the classic Shannon entropy with semantic similarity leads to 
 a more trustworthy measure. Lastly\, we apply Bayesian optimization to tra
 nslation reranking\, which uses kernel similarity to efficiently search fo
 r high quality translations.\n\n"You can also join us on Zoom":https://cam
 -ac-uk.zoom.us/j/83400335522?pwd=LkjYvMOvVpMbabOV1MVTm8QU6DrGN7.1\n
LOCATION:Lecture Theatre 2\, Computer Laboratory\, William Gates Building
END:VEVENT
END:VCALENDAR