Skip to content
Tutorials ▸

Neural networks, explained by building one

Invent an alphabet that has never existed, draw it by hand, and train a neural network to read your handwriting — in your browser, with every number visible at every step.

A trained scorecard for one invented character, after 400 steps. Teal counts for the character, pink against it. Nobody drew this — the network worked it out, and you can see the letter in it.

By the end: You will have written a loss function and a training loop by hand, in Python, and watched a network learn to read characters you invented.

Start with part 01

The 10 parts

  1. 01 The problem Five handwritten a's. Why comparing pictures fails, why writing rules fails, and what a network does instead of both. what a neural network is · dataset
  2. 02 Your alphabet Invent characters that have never existed and draw a dataset of them by hand. Nothing here mentions yet what the network will receive. dataset · labels · training examples
  3. 03 Into numbers A grid laid over your drawing turns it into 256 numbers, and reshape gets a step of its own. pixels · reshape · features
  4. 04 The network Every character gets a scorecard — one number per square. The weights are shown as pictures, because with one layer they can be. weights · parameters · bias · nn.Linear
  5. 05 How wrong is it? Softmax as shares of 100 per cent, cross-entropy as surprise, and loss as the single number training tries to shrink. softmax · cross-entropy · loss
  6. 06 Learning Gradient descent without calculus, and why averaging over many drawings turns memorising into understanding. gradient descent · learning rate · epoch · overfitting
  7. 07 The whole loop Six stations of training drawn as one circuit you can step around, one station at a time. training loop · epoch
  8. 08 Read my writing Draw something it has never seen and watch it read your handwriting, stroke by stroke. inference · generalisation · held-out data
  9. 09 Break it Starve it, lie to it, and move a weight by hand. Three ways to wreck a network on purpose, and what each failure looks like. overfitting · learning rate · distributed representation
  10. 10 More layers A real two-layer network, and backpropagation as blame travelling backwards through it. hidden layer · ReLU · backpropagation