Snap each coordinate to a level

A journey through finite scalar quantisation with Botie.

Natural Language Processing · A journey with Botie

First visit? Learn the workshop controls

Botie's notebook

Could a Small Grid Replace a Learned Catalogue of Codes?

Part 1 of 3

A learned vector codebook introduces its own training difficulties. Some entries may be used rarely or never, and the system needs to learn both the representation and the catalogue of allowed replacements. We can ask whether the discrete structure can be specified more simply.

Sources for this history

Mentzer and colleagues (2023), Finite Scalar Quantization: VQ-VAE Made Simple

← Previous lesson: Codebook Quantisation Read the full lesson and derivations