Natural Language Processing
Tell two identical tokens apart
A journey through positional encoding with Botie.
Natural Language Processing · A journey with Botie
First visit? Learn the workshop controlsBotie's notebook
How Would a Model Know Which Word Came First?
Part 1 of 3
The same words can describe different events when we change their order. A collection of word representations alone does not distinguish who acted on whom. If our architecture no longer processes tokens one after another, we need another way to make their positions available.