Try this first

The line y = 8x + 24. Without drawing it, answer three things. What score does it predict for 5 hours of study? What does it say about a student who studies nothing? And if a student studies one extra hour, how much does the prediction change?

Two numbers describe every straight line

The answers are 64, 24, and 8. Each came from one part of the formula, and each part has a plain meaning.

y = m * x + b

b is where the line crosses the vertical axis. It is the prediction when x is zero. Here it says a student who does no studying scores 24.

m is the slope. It is how much the prediction moves when x goes up by one. Here it says an hour of study is worth 8 marks. Its sign tells you the direction and its size tells you the steepness.

Those two numbers are the entire line. There is nothing else to know about it. Every one of the infinitely many straight lines on a plot is some pair (m, b), and choosing a line is choosing two numbers.

0 4 8 0 50 100 x y change m, keep b b = 24, held the line pivots 0 4 8 0 50 100 x change b, keep m b moves the line slides
Two numbers, two independent jobs. m is how steeply the line climbs, b is where it crosses the axis. Every straight line there is comes from one such pair.

In code

function predict(m, b, x) {
  return m * x + b;
}

log(predict(8, 24, 5));    // 64
log(predict(8, 24, 0));    // 24
log(predict(8, 24, 6) - predict(8, 24, 5));   // 8

Three lines that mirror the three questions. The third one measures the slope by asking for two predictions one apart and subtracting, which is worth remembering. We use exactly that trick in section 4 to measure something we cannot work out directly.

The names you will meet elsewhere

In machine learning writing, m is called a weight and b is called a bias. Same numbers, different words. The names come from the neural network side of the subject, where a model has thousands of weights and it stops being natural to call them slopes.

Together they are the parameters of the model. This is the word that matters. The model is the shape, y = mx + b, and you choose that. The parameters are the numbers filling it in, and those are what training changes. When you read that a model has seventy billion parameters, it means seventy billion numbers like m and b, all found by the same kind of search we build in section 4.

What to watch

A straight line can only say one thing: a fixed amount of change per unit. Eight marks for the first hour, eight for the tenth, eight for the fortieth. Anything that levels off, or accelerates, cannot be described by two numbers. When a straight line fits badly, it is often because the world it is describing does not work that way, and no amount of adjusting m and b will help.

Watch out for reading b too literally as well. Our data has no student below one hour, so 24 is a guess about a region we never measured.

Exercises

  1. Draw y = 8x + 24 and y = 8x + 34. Then y = 8x + 24 and y = 12x + 24. Say in words what changing each number did to the picture.
  2. Find, by trial and error, a pair (m, b) that passes exactly through [2, 41] and [7, 84]. Then work out how you could have got it directly from those two points.
  3. Write predictAll(m, b, data) which returns the predicted score for every row. You will use it in every one of the next four lessons.
  4. What line describes a prediction that ignores study time completely and always guesses the same score? Give the values of m and b. This one is the baseline any real model has to beat.
PreviousNext