Try this first

Here is a function: f(x) = x * x. At x = 3, how steep is it?

You do not need calculus to answer. You have a computer that can evaluate f anywhere you like. Work out a way to measure the steepness, and try it, before reading on.

Steepness is rise over run

Take two points close together and see how much the output changed compared to how much the input changed.

function slopeAt(f, x, h = 0.001) {
  return (f(x + h) - f(x)) / h;
}

const sq = x => x * x;
log(slopeAt(sq, 3).toFixed(6));

You get 6.001. The true answer is 6, and you got it by asking the function two questions.

h is how far apart the two points are. The formula measures the average slope across that gap. Since the curve bends, the average across a gap is not exactly the slope at the point, and the answer is off by a little. Make h smaller and the gap matters less.

How small should h be?

Try a range of values.

for (const h of [1, 0.1, 0.01, 1e-4, 1e-6, 1e-8, 1e-10, 1e-13, 1e-15]) {
  log(h, slopeAt(sq, 3, h).toFixed(10));
}

The answer marches towards 6 as h shrinks, exactly as expected, and then at around 1e-13 it starts getting worse again. By 1e-15 it is visibly wrong.

That is not a mistake in the formula. It is the floating point numbers running out of room. f(x + h) and f(x) are almost identical, so subtracting them cancels nearly every digit they had in common, and what survives is mostly rounding noise. Then you divide that noise by a tiny number, which magnifies it.

So there is a sweet spot. Too large and the curve bends inside the gap. Too small and the arithmetic falls apart. Somewhere near 1e-5 or 1e-6 is usually a good compromise for ordinary sized numbers.

-16 -12 -8 -4 0 -8 -6 -4 -2 0 log₁₀ h log₁₀ error lowest error for this f, h ≈ 1e-8 the safe default rounding noise wins the curve bends
Measured in the browser’s own arithmetic. On the right the curve is bending inside the gap. On the left the subtraction is cancelling the digits the two values shared.

A better formula for the same cost

function slopeAt(f, x, h = 1e-5) {
  return (f(x + h) - f(x - h)) / (2 * h);
}

Instead of looking forward from the point, look the same distance either side and use the middle. Still two evaluations. The errors from the curve bending in each direction partly cancel, so this is considerably more accurate. Use this version from now on.

2 3 4 4 9 16 zoomed in x x ² run = h rise forward chord slope 7 centred chord slope 6 true tangent slope 6 h = 1, drawn large so the gap is visible
Both measurements cost two evaluations of f. Looking only forward overshoots, because the curve bends inside the gap. Looking the same distance either side lets the two bends cancel.

What to watch

You have seen this idea before. In lesson 3045 you measured the slope of a line by asking for two predictions one apart and subtracting. Same move, and for a straight line the answer is exact because a straight line does not bend.

Keep this function even after the next lesson gives us a faster way. Being able to measure a slope by brute force is how you check that a derivative you worked out by hand is correct, and that check saves a great deal of time in section 8.

Exercises

  1. Use slopeAt on Math.sin at 0, at Math.PI / 2, and at Math.PI. Do the three answers match the shape of the sine curve?
  2. Find the slope of sq at x = 0. What does the answer mean about that point on the curve?
  3. Apply slopeAt to x => Math.abs(x) at x = 0 with the two sided formula. What comes out, and is that a sensible answer to the question “which way is downhill”?
  4. Write slopeOfLoss(m, b, data) that measures how the loss changes when only m moves. You need a function of one variable to hand to slopeAt, so wrap the loss in a closure.
PreviousNext