Stop a useful weight growing forever

A journey through hebbian principal components with Botie.

Neural Networks · A journey with Botie

First visit? Learn the workshop controls

Botie's notebook

Could a Neuron Discover a Repeated Direction by Itself?

Part 1 of 3

Suppose a neuron repeatedly receives patterns with a shared direction of variation. We would like its weights to become sensitive to that direction without supplying a class label. Strengthening connections when input and output are active together offers a starting idea, but unchecked strengthening can make the weights grow without limit.

Sources for this history

Oja (1982), A Simplified Neuron Model as a Principal Component Analyzer

← Previous lesson: Self-Organising Maps Read the full lesson and derivations