Try first

You are predicting the price of a house. So far you have used floor area, and y = mx + b handled it.

Now you also know the number of bedrooms, the age of the building, and the walking distance to the station. Write down how you would extend the formula to use all four. Do it before reading on. Most people get this right and it is worth knowing that you did.

One weight per feature

Give every feature its own multiplier and add them all up.

price = w0 * area
      + w1 * bedrooms
      + w2 * age
      + w3 * distance
      + b

That is multiple linear regression, and it is the obvious extension. Four features, four weights, still one bias.

Each weight has a plain reading: how much the prediction moves when that feature goes up by one and everything else stays put. So w0 is price per square meter, w2 is price per year of age, and you would expect that one to be negative.

The bias is what is left when every feature is zero. For houses that describes nothing real, which is a hint about how much to read into it.

area, m² 62 × +3.26 bedrooms 2 × +10.41 age, years 35 × -1.33 km to station 1.1 × -18.26 + add them up 266 thousand predicted price plus a bias of 110 the age weight is negative, and so is the one for distance: both make a house cheaper
One weight per feature, and each weight reads as a price. This is the same model as before with the single number replaced by a list, which is the only change in the section.

The data

const houses = [
  { x: [ 62,  2, 35,  1.1], y: 245 },
  { x: [ 78,  3, 12,  0.6], y: 372 },
  { x: [ 45,  1, 48,  2.3], y: 168 },
  { x: [110,  4,  8,  1.8], y: 465 },
  { x: [ 95,  3, 22,  0.4], y: 430 },
  { x: [ 58,  2, 60,  3.1], y: 190 },
  { x: [130,  5, 15,  2.0], y: 520 },
  { x: [ 71,  2, 30,  0.9], y: 298 },
  { x: [ 88,  3, 41,  1.5], y: 340 },
  { x: [102,  4, 19,  1.2], y: 448 },
];

Area in square meters, bedrooms, age in years, kilometers to the station, price in thousands. The same row shape as every dataset so far: x is the features, y is the answer.

What has changed and what has not

Almost nothing has changed. The loss is still mean squared error. Training is still gradient descent. The gradient still comes from the chain rule the same way, and you will see in the next lesson that the formula barely moves.

What has changed is that you can no longer draw it. One feature gave a line on a flat plot. Two features would give a tilted plane in a three dimensional picture, which is still just about drawable. Four features gives something in five dimensional space that nobody can picture.

So the pictures stop, and the algebra carries on unchanged. That is the trade this section makes. It is why the vector notation from lesson 3038 was worth the trouble: it keeps working when your intuition runs out.

What to watch

Look at the numbers in the columns. Area runs from 45 to 130. Bedrooms run from 1 to 5. Distance runs from 0.4 to 3.1. From lesson 3054 you know the gradient for a weight is the residual times that feature, so the area weight will see gradients about thirty times larger than the distance weight.

One learning rate has to serve all four. This is the diagonal valley from lesson 3050 again, made worse, and scaling the features is now less of a nicety and more of a requirement.

Exercises

  1. Guess the four weights from the data before training anything. Write them down. You will compare them in lesson 3068.
  2. Which of the four do you expect to be negative? Which do you expect to matter least?
  3. Add a fifth feature, price per square meter, computed from the data. Explain why including it would be cheating.
  4. Print the spread of each column with the spread and column helpers from lesson 3036. Confirm the range problem described above.
PreviousNext