Try this first

From the last exercise you know the loss changes as you move m. Before building anything, predict the shape. If you plotted loss against m, with b held still, what would the curve look like? Draw it.

Build the controls

Add two sliders to playground.html, above the canvas.

Then wire them up in the script.

const mSlider = document.getElementById("m");
const bSlider = document.getElementById("b");
const readout = document.getElementById("readout");

function render() {
  const m = parseFloat(mSlider.value);
  const b = parseFloat(bSlider.value);

  view.xMin = 0; view.xMax = 9;
  view.yMin = 0; view.yMax = 110;

  clear();
  grid(10);
  for (const r of study) {
    const p = predict(m, b, r.x[0]);
    line(r.x[0], p, r.x[0], r.y, "#eaa");
    dot(r.x[0], r.y, "#38a");
  }
  drawLine(m, b, "#333");

  const l = loss(m, b, study);
  readout.textContent =
    "m " + m.toFixed(1) + "   b " + b.toFixed(1) +
    "   loss " + l.toFixed(2) + "   rmse " + Math.sqrt(l).toFixed(2);
}

mSlider.oninput = render;
bSlider.oninput = render;
render();

render reads both sliders, redraws everything, and prints the loss. oninput fires on every movement of the handle, so the picture and the number stay in step with your hand. Nothing here is new. It is the last four lessons, connected to a control you can move.

Now use it

Do these in order and watch the number, not the line.

  1. Leave b at 24 and sweep m slowly from 0 to 16. The loss falls, reaches a bottom, and climbs again. One valley, no second one.
  2. Get the loss as low as you can with the sliders. Write down your best m, b and loss.
  3. From your best position, move m by 0.1. Note how much the loss changes. Now move b by 0.5 and note that.
  4. Set m to 0 and try to fix the fit using only b. See how low you can get.

What you should have noticed

Step 1 gives a valley with a single bottom. That is the property that makes this problem easy, and the last lesson of section 4 is about what happens when a problem does not have it.

Step 3 is the important one. Near the bottom, small moves barely change the loss. Far from the bottom, the same size move changes it a lot. The steepness tells you how far you are from the answer, and it also tells you which way to go. You have been doing gradient descent by hand for the last few minutes. Section 4 replaces your hand with about six lines of code.

Step 4 should have been frustrating. With the slope wrong, no intercept saves the fit. The two parameters are tangled together, and the best b depends on what m is.

What to watch

The sliders have limits and step sizes that I chose. If the true best m were 20, you could never reach it, and the tool would quietly mislead you. Every search has a region it can look in and a resolution it can look at, whether or not anyone tells you what they are.

Exercises

  1. Add a third slider that adds noise to the data, and watch how much the best loss rises while the best line barely moves.
  2. Print the loss to six decimal places. Move m by 0.1 near the bottom and far from it, and write down the two changes. That ratio is what section 4 computes directly.
  3. Add a button that sets m and b to random values, so you can start from somewhere bad. Keep it. Section 4 needs a starting point and this makes a good one.
  4. You minimized by hand in two dimensions and it was fiddly. Estimate how long it would take with ten parameters, then with a thousand.
PreviousNext