Try first

Ten thousand customer records. Age, spending, visits per month, and nothing else. No labels, no categories, nobody has said which customers are which type.

Your manager asks what kinds of customer there are. Every method in this course so far needs a y column. There isn’t one. What can you still do?

Two questions, and only one of them needs answers

Everything up to here has been supervised learning. Each row came with a right answer, the model guessed, and the loss measured the gap. Regression and classification are both this.

The alternative is unsupervised learning: find structure in the data with nobody to tell you what is correct. Grouping similar rows together, finding which features move together, spotting rows unlike all the others.

The difficulty is not the algorithm. It is that there is no loss. With no right answer there is nothing to measure the model against, so you cannot rank two answers the way lesson 3044 insisted you should be able to. That changes how you work: you look at the output and judge it, instead of reading a number.

What this section is really about

Four methods, and none of them use gradient descent. That is the point of including them.

You now know one machinery very well: define a loss, take its derivative, walk downhill. It works on an enormous range of problems and it is not the only way to learn from data.

k-nearest neighbors   no training at all, keeps the data
k-means               grouping without labels
decision trees        a series of yes/no questions
ensembles             many weak models, combined

Each is worth knowing on its own terms. Together they also stop gradient descent looking like the definition of the subject.

Where each one is the right choice

Small tabular data. Trees and ensembles usually beat neural networks here, and it is not close. Most business data is a few thousand rows and thirty columns, and a gradient boosted tree is the standard answer.

You need to explain the decision. A tree is a list of questions a person can read. A network with 30000 weights is not, and in lending, hiring or medicine that difference can be a legal requirement.

Almost no data. Fifty rows will not train a network. Nearest neighbors works from the first row.

No labels. Clustering is the only option available.

Neural networks win decisively on images, audio, and text, where the input is large, raw, and has structure that hand made features cannot capture. That is a real and important category, and it is not everything.

Two things that stay the same

Nearly everything from sections 2 and 9 carries over regardless of the method.

Distance still needs scaled features. Lesson 3042’s problem does not care which algorithm is measuring the distance, and both k-nearest neighbors and k-means are built entirely on distance.

Overfitting still happens. A decision tree grown to full depth memorizes the training set exactly, the same way the degree 9 polynomial did. Train, validation and test splits apply unchanged.

These are properties of learning from data, not properties of gradient descent.

What to watch

With no loss to check, unsupervised results are easy to over-read. Ask any clustering method for four groups and it returns four groups, whether or not the data has any. It will not report that the request was unreasonable. Whether the groups mean anything is your judgment, and it is worth being suspicious of your own eagerness to see structure.

Exercises

1. For each of these, say supervised or unsupervised: spam detection, grouping news articles by topic, predicting tomorrow’s temperature, finding unusual credit card transactions.
2. You have 500 labeled rows and 50000 unlabeled ones. Suggest a way to use both.
3. Name a case where an explainable model would be preferred to a more accurate one. Then argue the other side.
4. Given customer data with no labels, write down three groupings you would expect to find. Keep the list for lesson 3106.