Try this first

Eight measurements. Study time in hours against exam score.

[1, 32]  [2, 41]  [3, 52]  [4, 55]
[5, 68]  [6, 71]  [7, 84]  [8, 88]

Sketch these on paper and draw the straight line you would use to predict a score from a study time. Take thirty seconds over it. Then answer a harder question: how would you convince someone your line is better than theirs?

Everyone draws a slightly different line

Put the data in the playground and look at it.

const study = [
  { x: [1], y: 32 }, { x: [2], y: 41 }, { x: [3], y: 52 }, { x: [4], y: 55 },
  { x: [5], y: 68 }, { x: [6], y: 71 }, { x: [7], y: 84 }, { x: [8], y: 88 },
];

view.xMin = 0; view.xMax = 9;
view.yMin = 20; view.yMax = 100;

clear();
grid(1);
for (const r of study) dot(r.x[0], r.y);

The points climb steadily but they do not sit on one line. Point 4 is low, point 3 is high. Any line you draw will miss most of them.

Now draw three candidates and see how similar they look.

function drawLine(m, b, color) {
  line(view.xMin, m * view.xMin + b, view.xMax, m * view.xMax + b, color);
}

drawLine(8, 24, "#c33");
drawLine(7.5, 27, "#38a");
drawLine(9, 20, "#4a4");

Three lines, all reasonable, all different. If someone insists the red one is right, you have no way to argue except by pointing and saying it looks wrong.

The move that makes this a solvable problem

Stop trying to decide which line is best. Instead, define a number that says how bad a line is, and then look for the line with the smallest number.

That swap is the whole method, and it is worth sitting with, because it is what turns a question of taste into a calculation. Three consequences follow immediately:

  • Two people with the same definition of badness always agree on which line wins. No opinion enters.
  • A computer can now search. It cannot judge a line, but it can compare two numbers a million times a second.
  • Whatever you choose as the definition of badness decides which line comes out on top. Change the definition and you change the answer.

The rest of this section builds that number. Section 4 does the searching.

What to watch

Notice that the third point holds. There is no line that is best in general, only a line that is best according to a rule you picked. When you later hear that a model was trained to minimize something, the interesting question is always what that something was, and what it quietly ignores.

Also notice we have stopped talking about labels like “apple” and started predicting a number. Those are different jobs. Predicting a number is called regression, and it is this section and the next two. Predicting a class comes back in section 6.

Exercises

  1. Draw your paper line in the playground by estimating its slope and intercept. How close is it to one of the three above?
  2. Add the point [9, 40] to the data and plot again. Would you move your line for it? Would you move it much? Hold your answer, because lesson 4 of this section is about exactly this.
  3. Write down a rule, in one sentence, for how badly a line fits these eight points. Anything you can compute counts. Compare it to what we build in the next three lessons.
PreviousNext