BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:CausaLM: Causal Model Explanation Through Counterfactual Language 
 Models - Amir Feder\, Technion - Israel Institute of Technology
DTSTART:20210506T100000Z
DTEND:20210506T110000Z
UID:TALK160228@talks.cam.ac.uk
CONTACT:Haim Dubossarsky
DESCRIPTION:Understanding predictions made by deep neural networks is noto
 riously difficult\, but also crucial to their dissemination. As all ML-bas
 ed methods\, they are as good as their training data\, and can also captur
 e unwanted biases. While there are tools that can help understand whether 
 such biases exist\, they do not distinguish between correlation and causat
 ion\, and might be ill-suited for text-based models and for reasoning abou
 t high level language concepts. A key problem of estimating the causal eff
 ect of a concept of interest on a given model is that this estimation requ
 ires the generation of counterfactual examples\, which is challenging with
  existing generation technology. To bridge that gap\, we propose CausaLM\,
  a framework for producing causal model explanations using counterfactual 
 language representation models. Our approach is based on fine-tuning of de
 ep contextualized embedding models with auxiliary adversarial tasks derive
 d from the causal graph of the problem. Concretely\, we show that by caref
 ully choosing auxiliary adversarial pre-training tasks\, language represen
 tation models such as BERT can effectively learn a counterfactual represen
 tation for a given concept of interest\, and be used to estimate its true 
 causal effect on model performance. A byproduct of our method is a languag
 e representation model that is unaffected by the tested concept\, which ca
 n be useful in mitigating unwanted bias ingrained in the data.
LOCATION:https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBd
 XVpOXFvdz09
END:VEVENT
END:VCALENDAR