Try first

Last section. Before reading on, write down what would make a good final problem for this course.

Your list probably includes: uses most of what you built, you can see whether it worked, and it is hard enough to fail in interesting ways. Here is one more that matters as much: you must be able to get the data without downloading anything.

Handwritten digits

Ten classes, images in, a label out. It has been the standard first serious problem in this field for thirty years, and it earns that position.

It needs almost everything from this course. Features from raw pixels, from section 2. Scaling. Multiple classes, extending section 6. A hidden layer, from section 7, because pixels are not linearly separable in any useful way. Backpropagation, from section 8. Train and test splits, from section 9.

And you can look at it. When it misclassifies a 4 as a 9 you can see the image and often agree with it, which is not true of a model predicting churn from thirty anonymous columns.

The standard dataset is MNIST: 70000 handwritten digits, each 28 by 28 pixels in gray. That is the 784 features mentioned back in lesson 3036, one per pixel.

Two honest adjustments

We are not downloading MNIST, because this course has run with no dependencies and no network access from lesson 3034 onwards, and it is going to finish that way.

So we generate our own digits, by drawing glyphs to a canvas with random variation and reading the pixels back. It is a genuine image classification problem with real variation, and it is not as hard as real handwriting.

And we use a 16 by 16 grid, giving 256 features rather than 784. The reason is plain JavaScript speed. A 784 by 128 network on 60000 images is billions of multiplications per epoch, and our matmul from lesson 3065 would take many minutes per epoch. At 16 by 16 with a few thousand images it trains in seconds, which means you can experiment instead of waiting.

Nothing about the method changes. Every formula works at any size. This is a compromise about your time, and lesson 3115 covers what would have to change to run the full version.

Generating the data

const gen = document.createElement("canvas");
gen.width = 16; gen.height = 16;
const gctx = gen.getContext("2d");

const FONTS = ["16px serif", "16px sans-serif", "15px monospace", "bold 15px sans-serif"];

function renderDigit(d) {
  gctx.fillStyle = "#fff";
  gctx.fillRect(0, 0, 16, 16);
  gctx.save();
  gctx.translate(8 + (Math.random() - 0.5) * 2.5, 8 + (Math.random() - 0.5) * 2.5);
  gctx.rotate((Math.random() - 0.5) * 0.5);
  const s = 0.85 + Math.random() * 0.35;
  gctx.scale(s, s);
  gctx.fillStyle = "#000";
  gctx.font = FONTS[Math.floor(Math.random() * FONTS.length)];
  gctx.textAlign = "center";
  gctx.textBaseline = "middle";
  gctx.fillText(String(d), 0, 0);
  gctx.restore();

  const px = gctx.getImageData(0, 0, 16, 16).data;
  const features = [];
  for (let i = 0; i < 256; i++) features.push(1 - px[i * 4] / 255);
  return features;
}

function makeDigitSet(perClass) {
  const data = [];
  for (let d = 0; d <= 9; d++)
    for (let i = 0; i < perClass; i++)
      data.push({ x: renderDigit(d), y: d });
  return shuffled(data);
}

const digits = makeDigitSet(300);
log(digits.length, "images,", digits[0].x.length, "features each");

Each call shifts, rotates and scales the glyph a little, and picks a random font. That variation is what stops the model memorizing four exact pictures per digit.

The pixel read takes only the red channel, since the image is gray. 1 - value/255 flips it so ink is 1 and paper is 0, which puts the features in 0 to 1 already. The scaling from lesson 3042 is done, and it is done by construction rather than by fitting anything.

Look at the data before modeling it

function drawDigit(features, ox, oy, cell = 8) {
  for (let r = 0; r < 16; r++) {
    for (let c = 0; c < 16; c++) {
      const v = Math.round(255 * (1 - features[r * 16 + c]));
      ctx.fillStyle = "rgb(" + v + "," + v + "," + v + ")";
      ctx.fillRect(ox + c * cell, oy + r * cell, cell, cell);
    }
  }
}

clear();
for (let i = 0; i < 40; i++) {
  drawDigit(digits[i].x, (i % 10) * 46 + 4, Math.floor(i / 10) * 46 + 4, 2.6);
}
log(digits.slice(0, 40).map(d => d.y).join(" "));

Always look at your data. It takes two minutes and catches things no metric will: labels off by one, blank images, a class that never renders. If the pictures do not match the printed labels, stop and fix that before touching the model.

What to watch

Be clear about what this dataset is and is not. Real handwriting varies far more than a rotated font does: different stroke thickness, broken lines, personal quirks, genuine ambiguity between 1 and 7. Our generator produces variation that is smaller and more regular.

So expect high accuracy here and expect it to drop when you draw digits by hand in lesson 3113. That gap is itself the lesson, and it is the most common way real models disappoint: the test set resembled the training set more than reality does.

Exercises

  1. Generate 40 images of the digit 8 and look at them. Is the variation enough?
  2. Add stroke thickness variation using gctx.lineWidth and strokeText.
  3. Compute the average image for each digit and draw all ten. Which pairs look most alike?
  4. Count how many of the 256 pixels are zero in every single image. What does that say about the input?
PreviousNext