Abstract

Latent variable models provide an effective way to specify prior knowledge and uncover the intermediate decision process of natural language generation. In this talk, we will go through two specific applications. The first one incorporates latent continuous variables into a dialogue generation model. The latent variable is trained to maximize the mutual information with neighboring utterances. We show the latent variable component is able to significantly enhance the connection between the generated response and its surrounding context, leading to a more engaging human-machine conversation. The second one explicitly models the content selection process with discrete latent variables. By lowering down the training variance with a variational autoencoder objective, the model is able to successfully decouple content selection from the black-box generation model on both sentence compression and data-to-text tasks, enabling us to control the content selection in an interpretable way.