Abstract

Abstract:

Transformer-based language models (LMs) have demonstrated the promise in solving more and more complicated tasks, yet alongside their advancements are growing concerns on their safety and reliability. These concerns primarily stem from our limited understanding of these LMs and the difficulty in interpreting their behaviors. In this talk, I will present our two recent projects towards forming a mechanistic understanding of LMs. In the first project (published at ACL ’24), we explain how Chain-of-Thoughts (CoT) elicit the arithmetic reasoning of LMs by looking into the neuron activation inside the models; in the second project (ongoing), we generalize the analysis to understand the mechanism of how LMs solve the structured code generation problem. Finally, I will conclude the talk by briefly sharing our other effort along the line of LM planning and interpretability.

Bio:

Ziyu Yao (https://ziyuyao.org/) is an Assistant Professor in the Department of Computer Science at George Mason University, where she co-leads the George Mason NLP group (https://nlp.cs.gmu.edu/). Her research topics cover LLMs, semantic parsing/code generation, model interpretability, and human-AI interaction. Her work has been funded by National Science Foundation, Virginia Commonwealth Cyber Initiative, Microsoft’s Accelerating Foundation Models Research Program, among others. She has regularly served as an area chair at top-tier NLP /AI conferences and was the Diversity & Inclusion Co-Chair at NAACL 2024 . Prior to George Mason, she graduated with a Ph.D. degree in Computer Science and Engineering from the Ohio State University in 2021, where she was awarded the prestigious Presidential Fellowship. She was also selected as a rising star in EECS by UC Berkeley in 2021.