Turn boundary probabilities into pieces

A journey through learning boundaries from examples with Botie.

Natural Language Processing · A journey with Botie

First visit? Learn the workshop controls

Botie's notebook

Could Labelled Sentences Teach Us Where to Split?

Part 1 of 3

Writing enough rules to cover every boundary decision is difficult. But we may already have sentences divided by people according to an agreed convention. Those examples contain evidence about which neighbouring characters and patterns tend to accompany a word boundary.

Sources for this history

Xue (2003), Chinese Word Segmentation as Character Tagging

← Previous lesson: Segmenting With a Hidden Model Read the full lesson and derivations