Abstract

Abstract:

Yiwen Chen’s study looks at generating lyrics in Mandarin that match well with both the melody and the tonal contour of the language. The approach uses mBART and treats lyrics generation as a sequence-to-sequence (seq2seq) task. Instead of generating lyrics directly from the melody, which is the usual way, the system uses scansion as an intermediate step—a contour representation that works with the melody. An advantage of this method is that it doesn’t need a parallel melody-lyrics dataset.

The study also runs an automatic evaluation of the system against competitors, introducing new metrics specific to lyrics. These metrics check how clear the lyrics are, how well they fit the melody and measure creativity through metrics such as variation. Different ways of implementing scansion are tested and compared to other lyrics generators. The top-performing system beats all others in matching lyrics to melodies, and outperforms large language models (LLMs) built specifically for this task.

Bio: Yiwen Chen is a postdoctoral research assistant at the Centre for Digital Music at Queen Mary University of London. She’s also a professional lyricist who has collaborated with music labels and game companies. Her work focuses on improving the performance of lyrics generators without relying on parallel datasets of aligned melody and lyrics that involve copyright issues.