Into numbers

A network cannot look at a drawing. It can only do arithmetic on numbers, so the first job is to turn ink into numbers — and the way that happens is cruder than most people expect.

Lay a 16 by 16 grid over the drawing. For each of the 256 squares, ask a single question: how much ink is in this square? Nothing about strokes, corners or curves. Just an amount, square by square.

What does the grid throw away?

That grid keeps almost nothing. It does not know the order you drew the strokes in, which way the pen travelled, how long you took, or that a line is a line. A drawing is now 256 amounts of ink and nothing else.

It is worth sitting with how little that is, because everything the network ever learns it learns from those 256 numbers. If two characters look identical once blurred onto a 16 by 16 grid, no amount of training will separate them. The grid is a decision, and it is the first place accuracy can be lost.

Why is each drawing one row of numbers?

Those 256 numbers are laid out as a square because the drawing was, but the arithmetic does not care. Flattened into one long row of 256, with the label sitting beside it, every drawing becomes a row in a table — and that table is what training actually consumes.