Abstract

The core ingredient of deep learning is the notion of distributed representation. This talk will start by explaining its theoretical advantages for modelling language, in comparison with non-parametric methods based on counting frequencies of occurrence of observed tuples of values (like with n-grams). The talk will then explain how having multiple levels of representation, i.e., depth, can in principle give another exponential advantage. Neural language models have been extremely successful in recent years but extending their reach from language modelling to machine translation is very appealing because it forces the learned intermediate representations to capture meaning, and we found that the resulting word embeddings are qualitatively different. Recently, we introduced the notion of attention-based neural machine translation, with impressive results on several language pairs, and we applied the same ideas to caption generation, i.e., generating a sentence associated with an input image, yielding impressive results that will conclude the talk.

Yoshua Bengio is, along with Geoff Hinton and Yann LeCun, one of the machine learning researchers who has led the recent deep learning renaissance. He has made significant contributions to the understanding of deep neural networks and his work has led to advances in numerous application areas including computer vision, artificial intelligence and natural language understanding. He has won numerous awards for his work and his research has been cited over 22,000 times.