BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Multi-Head State Space Model for Sequence Modeling - Yassir Fathul
 lah\, Speech Group\, Cambridge University Engineering Department
DTSTART:20221011T140000Z
DTEND:20221011T150000Z
UID:TALK183179@talks.cam.ac.uk
CONTACT:Dr Kate Knill
DESCRIPTION:Recently\, state space models (SSMs) have shown promising resu
 lts on sequence modeling tasks. However\, a potential challenge of existin
 g works is that SSMs are usually introduced or initialized in a homogeneou
 s way\, encouraging the model to only capture similar temporal dynamics on
  different features. In this talk\, we propose a multi-head state space mo
 del (MSSM)\, in which parallel heads are introduced to learn different tem
 poral dynamics on sequence data. Furthermore\, we propose a novel variant 
 of the Transformer\, referred to as the Stateformer\, which combines MSSMs
  with attention. Experiments on large-scale automatic speech recognition (
 ASR) and language modeling tasks show the MSSM outperforming a range of at
 tention-based baselines. The Stateformer further improves performance\, ac
 hieving the state-of-the-art performance on the LibriSpeech ASR task.\n\nR
 esearch performed in Research Internship at Meta (AI Speech)\, California.
LOCATION: Hybrid: LT6\, First floor Baker building\, Engineering Dept or Z
 oom: https://eng-cam.zoom.us/j/81927138251?pwd=TVd3MXliV003dUdYVlFwU2NDWGp
 mdz09
END:VEVENT
END:VCALENDAR