The same code, with labels

You have now written two learning programs without using a single technical term. That was deliberate. The ideas came first. Now we attach the standard names, so that you can read other people’s writing and they can read yours.

Every name below points at a line you have already written.

Dataset

The examples you learn from. In our code, data. Each entry has an input and the correct output.

const data = [[1, 3], [2, 5], [3, 7], [4, 9]];

The input is usually called x or the features. The correct output is usually called y, the label, or the target. All three words for y mean the same thing.

Model

The shape of the guess. In our code, the predict function.

function predict(x) { return w * x + b; }

A model is not the answer. It is a family of possible answers. w * x + b describes every straight line there is. Choosing values for w and b picks one line out of that family.

When people say “a linear model” they mean this shape. When they say “a neural network” they mean a much larger family of shapes. The idea is identical: pick a family, then find the best member of it.

Parameters

The adjustable numbers inside the model. In our code, w and b.

These are what learning changes and they are all that gets saved when you save a model. Our model has two parameters. A large language model has hundreds of billions, but they play the same role.

Weight and bias are the conventional names for these two. The weight says how strongly the input affects the output. The bias shifts the output up or down regardless of the input.

Loss

One number saying how wrong the model currently is. In our code, the loss function.

total += error * error;

Lower is better. It is also called the cost or the objective. The particular version we used, average of squared errors, is called mean squared error. We will spend a whole lesson on why the errors are squared rather than just added.

The loss is the only thing that defines what “better” means. If you choose a loss that does not match what you actually care about, the model will get very good at the wrong thing. This is a common and expensive mistake.

Training

The process of changing the parameters to reduce the loss. In our code, the loop.

Two more terms live inside it. The learning rate is our rate, the size of each correction. An epoch is one complete pass over the dataset, which is one turn of the outer loop.

Inference

Using the trained model on new input. Calling predict(7) after training is inference. It is also called prediction or serving. No learning happens here, and the parameters do not change.

Check yourself

1. In the converter from lesson one, what was the model, what were the parameters, and what was the loss?
2. Our model has two parameters. If you also saved the learning rate, would that be a parameter of the model? Why not?
3. Someone says their model has 99 percent accuracy. Which of the terms above are they leaving out, and why might that matter?