BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Understanding Word Embeddings - Omer Levy\, Bar-Ilan University
DTSTART:20151013T130000Z
DTEND:20151013T140000Z
UID:TALK61259@talks.cam.ac.uk
CONTACT:Tamara Polajnar
DESCRIPTION:Neural word embeddings\, such as word2vec (Mikolov et al.\, 20
 13)\, have become increasingly popular in both academic and industrial NLP
 . These methods attempt to capture the semantic meanings of words by proce
 ssing huge unlabeled corpora with methods inspired by neural networks and 
 the recent onset of Deep Learning. The result is a vectorial representatio
 n of every word in a low-dimensional continuous space. These word vectors 
 exhibit interesting arithmetic properties (e.g. king - man + woman = queen
 ) (Mikolov et al.\, 2013)\, and seemingly outperform traditional vector-sp
 ace models of meaning inspired by Harris's Distributional Hypothesis (Baro
 ni et al.\, 2014). Our work attempts to demystify word embeddings\, and un
 derstand what makes them so much better than traditional methods at captur
 ing semantic properties.\n\nOur main result shows that state-of-the-art wo
 rd embeddings are actually "more of the same". In particular\, we show tha
 t skip-grams with negative sampling\, the latest algorithm in word2vec\, i
 s implicitly factorizing a word-context PMI matrix\, which has been thorou
 ghly used and studied in the NLP community for the past 20 years. We also 
 identify that the root of word2vec's perceived superiority can be attribut
 ed to a collection of hyperparameter settings. While these hyperparameters
  were thought to be unique to neural-network-inspired embedding methods\, 
 we show that they can\, in fact\, be ported to traditional distributional 
 methods\, significantly improving their performance. Among our qualitative
  results is a method for interpreting these seemingly-opaque word-vectors\
 , and the answer to why king - man + woman = queen.\n\nBased on joint work
  with Yoav Goldberg and Ido Dagan.
LOCATION:FW11\, Computer Laboratory
END:VEVENT
END:VCALENDAR