Understanding Machine Learning

Introduction

Supervised learning is how most classification models get built. It's basically how we learn too: look at examples, notice the patterns, get quizzed, and fix what we got wrong.

For a machine the loop looks like this:

see labeled data → build a model → train the model → make predictions → evaluate

The key word is supervised. We give the model a pile of examples where we already know the answer, and it works out a rule that matches them. Ideally that rule still works on stuff it hasn't seen.

To keep it concrete we'll build one small model start to finish: a spam classifier. Every email is just two numbers so we can plot it on a chart:

• links — how many links the email has (0–20)
• caps ratio — how much of the text is ALL CAPS (0–1)

Every demo below runs on this same data.

The Machine Learning Process for Supervised Learning

1. Data

Data comes first, because the data is where the patterns are. Supervised learning needs labeled data — input → output pairs, sometimes called structured data. It's like how kids learn names: point at an animal, say "dog" enough times, and they start to generalize.

Labeled data is really just a table. Each row is one email (the inputs) plus the answer we want the model to learn (the label). Here that's two numbers, links and caps ratio, paired with spam or not spam.

Add or remove rows below to play with it. This is the part a human does by hand, before any model exists. It's the only step where we fill in the answers ourselves.

Cool Note: this table feeds every demo on the rest of the page. The model, the training, the predictions, the scores — they all learn from the rows you see here. Change it and scroll down to watch everything react.

2. Model

The model is the rule that turns features into a prediction. We'll use a perceptrona linear classifier that learns a set of weights and splits the feature space with a single straight line. It's about as simple as models get: draw one straight line, call everything on one side spam.

It does this by adding up a weighted score:

score = w₁ · links + w₂ · caps + b

If the score is ≥ 0 it says spam, otherwise not spam. The weights w₁ and w₂ say how much each feature matters, and the bias b slides the line up or down. Those three numbers are the whole model.

Drag the dials to move the line yourself. The shaded areas show what the model would predict everywhere, and the circled points are the ones it's getting wrong. See how close you can get to splitting the two groups.

3. Training

Setting those dials by hand is fine for three of them, but real models have thousands or millions. Training is just letting the model find good weights on its own from the data.

The perceptron's rule is simple. Start with the weights at zero and go through the examples one at a time:

• If it got the example right, leave the weights alone.
• If it got it wrong, push every weight a bit toward the answer.

That "a bit" is the important part. Each wrong guess updates a weight like this:

wᵢ ← wᵢ + η · (actual − predicted) · featureᵢ

ηeta is the learning rate — the learning factor that controls how big each step is. (actual − predicted) is just +1 or −1 depending on which way it missed, so the weight moves toward fixing that point, scaled by the feature and by η. Small η means tiny, careful steps that take a while; big η means fast steps that can overshoot and bounce around. The demo uses η = 0.1.

One full pass over the data is an epochone complete sweep through every training example. Keep going until a whole epoch passes with zero mistakes — at that point the data is split cleanly and the model has converged.

Hit Train and watch the line snap into place as the errors drop to zero.

4. Prediction

Once training's done, the weights are locked in and the model is ready to use. Prediction (or inferencerunning a trained model on new inputs to produce predictions) is just dropping a new, unlabeled email into that same score formula and seeing which side of the line it lands on.

The model below is already trained on the data from section 1. Pick one of the three incoming emails — the model grabs its two features, plots it, and classifies it by which region it falls in. It's never seen these exact emails; it's just going off the line it learned.

5. Evaluation

There's a catch. If we only ever test the model on the emails it trained on, it'll look perfect — it could just be memorizing them (that's overfittingwhen a model fits its training data closely but fails to generalize to new, unseen data). To see if it actually generalizes, we hold some labeled data back as a test set, train on the rest, and score it on the data it never saw.

Accuracy (correct ÷ total) is the obvious number, but it hides how the model is wrong. A confusion matrix splits the results into four buckets:

• True positive — spam correctly caught
• True negative — good email correctly let through
• False positive — good email wrongly flagged as spam (annoying)
• False negative — spam that slipped into the inbox

The demo trains on ~70% of the data and scores the held-out ~30% (the dashed points). Hit New split to shuffle which emails get held out and watch the accuracy move around — one number depends a lot on which data you happened to test on.

End -- Naming things is hard

So that's the whole loop on one tiny problem: grab labeled data, pick a model, train it to find good weights, predict with it, and evaluate it on data it never saw. Spam filters, image classifiers, all of it follows this same skeleton — they just swap our straight line for much bigger models and our two features for thousands.

If you want to keep going, two easy next steps: swap the perceptron's hard yes/no for logistic regression (which gives you a probability instead), and add more than two features so the boundary lives in higher dimensions.