BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:SafetyKit: First Aid for Measuring Safety in Open-domain Conversat
 ional Systems - Verena Rieser\, Heriot-Watt University
DTSTART:20230126T110000Z
DTEND:20230126T120000Z
UID:TALK196105@talks.cam.ac.uk
CONTACT:Panagiotis Fytas
DESCRIPTION:The social impact of natural language processing and its appli
 cations has received increasing attention. In this position paper\, we foc
 us on the problem of safety for end-to-end conversational AI. We survey th
 e problem landscape therein\, introducing a taxonomy of three observed phe
 nomena: the Instigator\, Yea-Sayer\, and Impostor effects. We then empiric
 ally assess the extent to which current tools can measure these effects an
 d current systems display them. We release these tools as part of a “fir
 st aid kit” (SafetyKit) to quickly assess apparent safety concerns. Our 
 results show that\, while current tools are able to provide an estimate of
  the relative safety of systems in various settings\, they still have seve
 ral shortcomings. We suggest several future directions and discuss ethical
  considerations.
LOCATION:https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBd
 XVpOXFvdz09
END:VEVENT
END:VCALENDAR