Recover the boundaries behind the characters

A journey through segmenting with a hidden model with Botie.

Natural Language Processing · A journey with Botie

First visit? Learn the workshop controls

Botie's notebook

Could Hidden Labels Explain the Boundaries We Cannot See?

Part 1 of 3

A dictionary cannot directly list every new name or unfamiliar word. We could instead ask what role each visible character plays: does it begin a word, continue one, end one, or stand alone? Those roles are not written in the input, so they must be inferred.

Sources for this history

Rabiner (1989), A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition

← Previous lesson: The Word Lattice Read the full lesson and derivations