Natural Language Processing

Learn from the nearby words

A journey through word2vec with Botie.

Natural Language Processing · A journey with Botie

First visit? Learn the workshop controls

Botie's notebook

Could a Small Prediction Task Learn Useful Word Relationships?

Part 1 of 3

Learning a complete language model can be expensive when our immediate goal is only a useful vector for each word. We would like a simpler task that still forces those vectors to capture relationships in the text.

Sources for this history

Mikolov and colleagues (2013), Efficient Estimation of Word Representations in Vector Space
Mikolov and colleagues (2013), Distributed Representations of Words and Phrases and Their Compositionality

← Previous lesson: The Embedding Layer Read the full lesson and derivations