Abstract

Abstract:

This talk explores alternatives to autoregressive Large Language Models (LLMs), with a particular focus on discrete diffusion models. The presentation covers recent advances in non-autoregressive approaches to text generation, reasoning, and planning tasks. Key developments discussed include Reparameterized Discrete Diffusion Models (RDMs), which show promising results in machine translation and error correction, and applications of discrete diffusion to complex reasoning tasks like countdown games, Sudoku, and chess. The talk also examines sequence-to-sequence text diffusion models, as well as the novel Diffusion of Thoughts (DoTs) framework for chain-of-thought reasoning. These non-autoregressive approaches demonstrate competitive performance while offering potential advantages in terms of parallel processing and flexible generation patterns compared to traditional autoregressive models.

A short CV:

https://ikekonglp.github.io/lingpenk_cv.pdf

Bio:

Lingpeng Kong is an assistant professor in the Department of Computer Science, the University of Hong Kong. From 2017-2020, he was a (senior) research scientist at Google DeepMind. He received his Ph.D. in Artificial Intelligence from the School of Computer Science, Carnegie Mellon University. His work lies at the intersection of natural language processing (NLP) and machine learning (ML), with a focus on representation learning, structured prediction, and generative models. He has served as the (senior) area chairs for top NLP and ML conferences such as ICLR , NeurIPS, ICML , EMNLP, and NAACL . He has also served as an action editor for Transactions on Machine Learning Research (TMLR) and ACL rolling review (ARR). His research was recognized with outstanding paper awards from ACL and EACL .

A photo for the event web page: https://ikekonglp.github.io/me_new.jpeg