Try this first

Three points: A = [2, 3], B = [5, 7], C = [6, 3].

Is B or C closer to A? Answer from the numbers alone, before you calculate anything.

Distance is the length of the difference

B is 3 across and 4 up from A. C is 4 across and 0 up. Pythagoras gives 5 for B and 4 for C, so C is closer. Plenty of people say B, because the gap to B is diagonal and diagonal gaps read as shorter than they are.

You already have the two pieces. Subtracting gives the difference vector. norm gives its length. So distance is one line.

function distance(a, b) {
  return norm(sub(a, b));
}

log(distance([2, 3], [5, 7]));   // 5
log(distance([2, 3], [6, 3]));   // 4

Nothing about this line mentions two dimensions. Hand it two arrays of length 784 and it still works, and it still means the same thing. This is why we bothered with sub and norm as separate functions rather than writing the distance formula out directly.

Using it to answer a question

Now something a program can actually do. Given a new item, find the item in the dataset it most resembles.

function nearest(point, data) {
  let best = null;
  let bestD = Infinity;
  for (const row of data) {
    const d = distance(point, row.x);
    if (d < bestD) {
      bestD = d;
      best = row;
    }
  }
  return { row: best, d: bestD };
}

const mystery = [195, 7.5, 0.85];
log(JSON.stringify(nearest(mystery, fruit)));

Walk every row, measure the distance, keep the smallest one seen so far. bestD starts at Infinity so the first row always beats it. That is the entire method.

Notice what you just built. If you take the label of the nearest row and call it the answer, you have a working classifier, and it has no training step at all. We come back to this properly in section 10.

What to watch

Run nearest on the fruit data and look closely at which row wins. Then look at the numbers.

weight difference   25      squared: 625
diameter difference  0.4    squared:   0.16
orangeness diff      0.03   squared:   0.0009

The sum is 625.16. Weight is 99.97 percent of it. Whatever the function is called, it is measuring weight and rounding everything else away. Orangeness, the one feature that actually separates apples from oranges, has no say at all.

This is not a flaw in the distance formula. It is a consequence of measuring weight in grams and orangeness on a scale of 0 to 1. The last lesson in this section fixes it.

One more question worth answering now: why square and then take a square root, instead of just adding up the sizes of the differences? Adding sizes is a real distance too, called Manhattan distance, and it is sometimes the better choice. We use the squared version because it matches straight line distance on a plot and because it is smooth, which means it has a derivative everywhere. In section 4 we will need that derivative.

Exercises

  1. Check by hand that distance(a, b) and distance(b, a) agree, and say which line of norm guarantees it.
  2. Find the two most similar rows in the fruit dataset by comparing every pair. How many comparisons did you make for six rows? How many would you make for a thousand?
  3. Run nearest using only the orangeness column, by passing arrays of length one. Compare the answer to the full three feature version. The shorter one is more accurate, and that should bother you enough to make the next two lessons worth reading.
PreviousNext