Neural Networks

Keep the average signal steady

A journey through dropout with Botie.

Neural Networks · A journey with Botie

First visit? Learn the workshop controls

Botie's notebook

What Happens When a Neuron Cannot Rely on the Same Partners?

Part 1 of 3

A large network can learn combinations that work very well on its training examples but depend too closely on their particular details. We would like useful signals to remain useful when some of their usual partners are absent.

Sources for this history

Srivastava and colleagues (2014), Dropout: A Simple Way to Prevent Neural Networks from Overfitting

← Previous lesson: Training a Small Network Read the full lesson and derivations