University of Cambridge > Talks.cam > NLIP Seminar Series > Generating Natural-Language Video Descriptions using LSTM Recurrent Neural Networks

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Generating Natural-Language Video Descriptions using LSTM Recurrent Neural Networks

Download to your calendar using vCal

If you have a question about this talk, please contact Kris Cao .

We present a method for automatically generating English sentences describing short videos using deep neural networks. Specifically, we apply convolutional and Long Short-Term Memory (LSTM) recurrent networks to translate videos to English descriptions using an encoder/decoder framework. A sequence of image frames (represented using deep visual features) is first mapped to a vector encoding the full video, and then this encoding is mapped to a sequence of words. We have also explored how statistical linguistic knowledge mined from large text corpora, specifically LSTM language models and lexical embeddings, can improve the descriptions. Experimental evaluation on a corpus of short YouTube videos and movie clips annotated by Descriptive Video Service demonstrate the capabilities of the technique by comparing its output to human-generated descriptions.

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Tim Nickels Room, CUED Dr Hanno Balz on 'Hostile Takeover: A Political History of the Colour Red' Keita Tamura Lab @ PDN

Other talks

TBC Group Work Feedback and the circumgalactic medium Group Presentation 1 Title to be confirmed Director's Welcome
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity