Try first

Back to the fruit from lesson 3036. You want a program that says “apple” or “orange”.

You have a working linear model. Feed it the features, get a number out. Now: how do you turn that number into a word? Write down your rule before reading on.

The obvious rule, and what it costs

Nearly everyone writes the same thing. Label apples 0 and oranges 1, train the model to predict that number, then round.

const label = predict(w, b, x) > 0.5 ? "orange" : "apple";

This works, and it is roughly what we are going to build. But it papers over a real change in the problem, and the change is worth naming before we start.

Three ways the question is different

The output is not on a scale. Predicting 460 for a house worth 465 is a good answer. Predicting 0.5 when the answer is apple is not half right, it is useless. Between 0 and 1 there are no intermediate fruits.

Being wrong is not symmetric. Squared error says missing by 0.4 is worse than missing by 0.2, four times over. For a class, what matters is which side of the line you came down on. A confident wrong answer and a hesitant wrong answer are both wrong, but they are not equally bad, and squared error ranks them the wrong way round.

The number 0.5 is doing a lot of work. Nobody chose it. It arrived because the labels were 0 and 1 and rounding felt natural. Lesson 3076 shows why that choice deserves more thought than it usually gets.

What we actually want

Not “apple” or “orange” straight away, and not an unbounded number either. What we want is a probability: this is an orange with confidence 0.87.

That is more useful than a bare label for three reasons. It says how sure the model is, so you can send borderline cases to a person. It lets you move the cut off later without retraining. And it gives you something smooth to run gradient descent on, which the bare label does not.

So the job for this section is: get a number between 0 and 1 out of a linear model, and find a loss that treats it as a probability. Those are the next three lessons.

The data

const points = [];
for (let i = 0; i < 60; i++) {
  const cls = i % 2;
  const cx = cls ? 6.5 : 3.5;
  const cy = cls ? 6.0 : 4.0;
  points.push({ x: [cx + (Math.random() - 0.5) * 3,
                    cy + (Math.random() - 0.5) * 3], y: cls });
}

view.xMin = 0; view.xMax = 10; view.yMin = 0; view.yMax = 10;
clear(); grid(1);
for (const p of points) dot(p.x[0], p.x[1], p.y ? "#e80" : "#38a");

Two clouds with a bit of overlap. Two features so you can see everything, and y is 0 or 1 rather than a word, because the arithmetic needs numbers.

Look at the overlap in the middle. No straight line separates these perfectly, and any honest model has to be uncertain in that region. That is what the probability is for.

What to watch

Labeling apple 0 and orange 1 is arbitrary and harmless with two classes. With ten classes, labeling them 0 to 9 is actively wrong, because it tells the model that 8 is close to 9 and far from 1, which is nonsense for digits. Section 11 deals with that.

Exercises

1. Draw the line you would use to separate the two clouds. Estimate its m and b.
2. Count how many points your line gets wrong. Can you do better with a different line?
3. Change the two centers to be further apart and regenerate. At what separation does the overlap disappear?
4. Suppose the classes were “has the disease” and “does not”. Which mistake would you rather make? Hold that thought until lesson 3076.