Try this first

Here are two apples and one orange, as weight in grams and orangeness from 0 to 1.

apple  A = [180, 0.15]
apple  B = [220, 0.10]
orange C = [182, 0.90]

Which two are most alike? Answer as a person. Then work out distance(A, B) and distance(A, C) and see whether the function agrees with you.

The function disagrees, and it is not wrong

You said A and C are furthest apart, because one is an apple and one is an orange. The function says distance(A, B) is 40 and distance(A, C) is 2.15, so it thinks A and C are nearly the same thing.

Both answers follow correctly from their inputs. Weight moves in steps of tens. Orangeness moves in steps of hundredths. When you square them and add, the tens win every time, and the units were never comparable to begin with. Adding grams squared to orangeness squared is arithmetic on things that have nothing to do with each other.

Since we cannot make the units comparable, we make the numbers comparable instead. Put every feature on the same numerical range before measuring anything.

Min-max scaling

Squeeze each column into the range 0 to 1. The smallest value in a column becomes 0, the largest becomes 1, everything else lands proportionally in between.

function minMaxScaler(data) {
  const n = data[0].x.length;
  const lo = [], hi = [];
  for (let i = 0; i < n; i++) {
    const col = column(data, i);
    lo.push(Math.min(...col));
    hi.push(Math.max(...col));
  }
  return function (x) {
    return x.map((v, i) => hi[i] === lo[i] ? 0 : (v - lo[i]) / (hi[i] - lo[i]));
  };
}

Read the shape of this before the arithmetic. It takes a dataset and returns a function. The returned function remembers the lo and hi it measured, so every item you push through it is treated identically.

The arithmetic itself is one line. Subtract the minimum so the column starts at 0, then divide by the width of the column so it ends at 1. The hi[i] === lo[i] check catches a column where every value is the same, which would otherwise divide by zero.

Standardization

The other common choice. Center each column on its mean and divide by its standard deviation, so a value becomes “how many typical gaps above average is this”.

function standardScaler(data) {
  const n = data[0].x.length;
  const mu = [], sd = [];
  for (let i = 0; i < n; i++) {
    const col = column(data, i);
    const m = col.reduce((s, v) => s + v, 0) / col.length;
    const varc = col.reduce((s, v) => s + (v - m) * (v - m), 0) / col.length;
    mu.push(m);
    sd.push(Math.sqrt(varc) || 1);
  }
  return x => x.map((v, i) => (v - mu[i]) / sd[i]);
}

The || 1 handles a constant column, where the standard deviation is 0. Dividing by 1 leaves those values at 0 after centering, which is the sensible answer.

Try it

const scale01 = minMaxScaler(fruit);
const scaled = fruit.map(row => ({ x: scale01(row.x), y: row.y }));

log("before", distance(fruit[0].x, fruit[3].x).toFixed(3));
log("after ", distance(scaled[0].x, scaled[3].x).toFixed(3));

const mystery = scale01([195, 7.5, 0.85]);
log(JSON.stringify(nearest(mystery, scaled)));

Run nearest before and after. Before scaling it picks whichever fruit weighs about the same. After scaling it picks an orange, which is the answer a person would give.

What to watch

Fit the scaler once, on your dataset, and reuse it. Building a fresh scaler for a single new item is meaningless, because one item has a minimum and maximum equal to itself and every feature collapses to 0. This is the most common way people break this step, and it produces no error message.

The two methods behave differently under outliers. One fruit weighing 2000g drags hi up, and min-max squashes every other item into the bottom sliver of the range. Standardization is less affected, but its output is not bounded, so a value can come out at 6 or at -4.

Notice something new here. The scaler holds numbers measured from the data, and those numbers change the answers. It is not the model, but it was learned from the data all the same, and it has to be saved and shipped alongside the model. Section 9 covers why it must be fitted on the training set alone.

Exercises

  1. Scale the fruit data with both scalers and print all six rows for each. Say which one you would rather look at, and why.
  2. Add a 2000g fruit to the dataset, refit both scalers, and print the results. Confirm the difference described above with your own numbers.
  3. Plot the fruit with scatter before and after scaling, using weight and orangeness. The shape of the cloud stays the same, only the axis numbers change. Explain why that had to be true.
  4. Apply scale01 to an item whose weight is higher than anything in the dataset. What value comes out, and is that a problem? Answer before section 9, where you meet data the model has never seen.
PreviousNext