Natural Language Processing

Let the query choose what contributes

A journey through attention with Botie.

Natural Language Processing · A journey with Botie

First visit? Learn the workshop controls

Botie's notebook

Why Must a Translator Remember the Whole Sentence in One Summary?

Part 1 of 3

Imagine translating a long sentence after reading it once, while being allowed to keep only one short note. Different output words may depend on different parts of the original sentence. The note has to preserve everything that might later matter.

Sources for this history

Bahdanau, Cho and Bengio (2014), Neural Machine Translation by Jointly Learning to Align and Translate
Vaswani and colleagues (2017), Attention Is All You Need

← Previous lesson: Positional Encoding Read the full lesson and derivations