What Is Machine Learning?

Ordinary programming is telling a computer exactly what to do. If you want to know whether someone can vote, you write the rule:

def can_vote(person):    return person.age >= 18 and person.is_registered

This is a good program. It is fast, it is exactly right, anyone can read it, and when the law changes you change one line. Do not use machine learning for this. If you can write the rule down, write the rule down.

Now try: is this message spam?

You start writing rules. Lots of links is suspicious — unless it is a newsletter. ALL CAPS is suspicious — unless it is a short excited message from a friend. The word “free” is suspicious — unless the email is about free shipping on something you ordered.

Every rule you write needs an exception, and the exceptions need exceptions. Try it below: each step adds the single best rule that can be added, chosen automatically, so you are seeing hand-written rules at their absolute best.

Two things are worth noticing, and the second is the one that matters.

First, the returns collapse. The first rule buys you thirty percentage points. By rule six the gains are down to fractions of a point, and rules seven and eight buy literally nothing — there is no rectangle left worth adding. The effort per unit of improvement climbs until it is not worth paying.

Second, look at the shape. Each rule is a rectangle, because that is what an if-statement about two numbers draws. But the real boundary between spam and not-spam runs diagonally — three links in a short message is suspicious, three links in a long newsletter is not. A staircase of rectangles can get closer and closer to a diagonal and never become one.

The dashed line is what a model learns from the same data. It is one line rather than eight rules, nobody wrote it, and it does better than the rules ever manage.

This trade is not free, and it is worth knowing the price before you start.

You need examples, and lots of them.The rule for voting needed nobody’s data. A spam filter needs thousands of messages that somebody already labelled.

You lose the ability to read it. You can read age >= 18 and know exactly what it does. A learned model is a pile of numbers, and explaining any single decision is its own hard problem.

It is never exactly right. Notice that even the learned line does not get everything — some of these messages are ambiguous, and no method recovers those. Machine learning trades certainty for coverage.