BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:What do sentence transformers know\, and how can we find out? - Se
 bastian Pado\, Stuttgart University
DTSTART:20240229T110000Z
DTEND:20240229T120000Z
UID:TALK212497@talks.cam.ac.uk
CONTACT:Panagiotis Fytas
DESCRIPTION:Transformer architectures specialised for producing full-sente
 nce\nrepresentations\, notably SBERT\, often achieve better performance on
 \ndownstream tasks than sentence embeddings extracted from vanilla BERT.\n
 However\, compared to Vanilla transformers\, we still have a limited\nunde
 rstanding of which linguistic properties of the inputs are\nrepresented we
 ll (or less well) within these models.\n\nIn my presentation\, I will repo
 rt on two angles from which we have\nanalyzed SBERT: (a)\, a black-box tes
 ting approach where we build\nminimal pairs of synthetic sentences to obse
 rve and analyze\ndifferences in the model's predictions [1]\; and (b)\, a 
 white-box\ntesting approach where we extend the Integrated Gradients attri
 bution\nmethod to the Siamese case. This permits us to decompose model\npr
 edictions on arbitrary input in terms of the contributions of\nindividual 
 token pairs [2\,3].\n\n[1] Dmitry Nikolaev and Sebastian Padó.\n    Repre
 sentation biases in sentence transformers.\n    Proceedings of EACL. Dubro
 vnik\, Croatia\, 2023.\n    https://aclanthology.org/2023.eacl-main.268   
  \n\n[2] Lucas Möller\, Dmitry Nikolaev and Sebastian Padó.\n    An Attr
 ibution Method for Siamese Encoders.\n    Proceedings of EMNLP. Singapore\
 , 2023.\n    https://aclanthology.org/2023.emnlp-main.980\n    \n[3] Lucas
  Möller\, Dmitry Nikolaev and Sebastian Padó.\n    Approximate Attributi
 ons for Off-the-Shelf Siamese Transformers.\n    Proceedings of EACL. St J
 ulian's\, Malta\, 2024.\n    https://arxiv.org/abs/2402.02883
LOCATION:GR06/07 | Faculty of English\, 9 West Road\, CB3 9DP
END:VEVENT
END:VCALENDAR