BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Mechanistic Understanding of Language Models in Arithmetic Reasoni
 ng and Code Generation - Ziyu Yao\, George Mason University
DTSTART:20241205T140000Z
DTEND:20241205T150000Z
UID:TALK222286@talks.cam.ac.uk
CONTACT:Tiancheng Hu
DESCRIPTION:Abstract:\n\nTransformer-based language models (LMs) have demo
 nstrated the promise in solving more and more complicated tasks\, yet alon
 gside their advancements are growing concerns on their safety and reliabil
 ity. These concerns primarily stem from our limited understanding of these
  LMs and the difficulty in interpreting their behaviors. In this talk\, I 
 will present our two recent projects towards forming a mechanistic underst
 anding of LMs. In the first project (published at ACL’24)\, we explain h
 ow Chain-of-Thoughts (CoT) elicit the arithmetic reasoning of LMs by looki
 ng into the neuron activation inside the models\; in the second project (o
 ngoing)\, we generalize the analysis to understand the mechanism of how LM
 s solve the structured code generation problem. Finally\, I will conclude 
 the talk by briefly sharing our other effort along the line of LM planning
  and interpretability.\n\n \n\nBio:\n\nZiyu Yao (https://ziyuyao.org/) is 
 an Assistant Professor in the Department of Computer Science at George Mas
 on University\, where she co-leads the George Mason NLP group (https://nlp
 .cs.gmu.edu/). Her research topics cover LLMs\, semantic parsing/code gene
 ration\, model interpretability\, and human-AI interaction. Her work has b
 een funded by National Science Foundation\, Virginia Commonwealth Cyber In
 itiative\, Microsoft’s Accelerating Foundation Models Research Program\,
  among others. She has regularly served as an area chair at top-tier NLP/A
 I conferences and was the Diversity & Inclusion Co-Chair at NAACL 2024. Pr
 ior to George Mason\, she graduated with a Ph.D. degree in Computer Scienc
 e and Engineering from the Ohio State University in 2021\, where she was a
 warded the prestigious Presidential Fellowship. She was also selected as a
  rising star in EECS by UC Berkeley in 2021.\n\n
LOCATION:https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBd
 XVpOXFvdz09
END:VEVENT
END:VCALENDAR