Chapter 12: Machine Learning
Every morning, at a vegetable stall in the market, the seller tips out a crate of tomatoes and sorts them into two heaps. The firm red ones go to the front of the stall. The soft or pale ones are kept aside to be sold cheap by evening. She does this in a few seconds, without weighing anything and without looking anything up.
Ravi, who buys vegetables there on his way home, wanted to write a program that could do the same job. He began with one rule: if the tomato is red, keep it. That failed at once, because some red ones were soft. So he added a rule about size, then one about spots, then one about the stem. Each new rule fixed one tomato and spoilt another. After a week he had a long list of rules and a program that still got it wrong.
The seller never learnt a list of rules. She learnt by looking at thousands of tomatoes. That is exactly the idea this chapter is about.
Diagram
Fig. 12.0-1 – photo of tomatoes spread out on a tray at a vegetable stall, some firm and deep red, others pale or soft, with a hand picking one up
This chapter will answer all five of these questions as you work through it.
12.1 What Machine Learning Is
Think again about the tomato stall. The seller does a job that she cannot fully explain in words. If you asked her to write down exactly how she picks the good ones, she would find it hard — and yet she almost never gets it wrong.
Computers have the opposite problem. A computer follows instructions exactly, but only the instructions it is given. So if a job cannot be written down as instructions, how do you get a computer to do it?
That question is where this chapter begins.
12.1.1 Rules Written by Hand, and Rules Learnt from Examples
In the programs you have written so far, you decided the rules. You worked out what the program should do in each situation, wrote those steps in Python, and the computer carried them out. Whatever you put into a program is called its input, and whatever it gives back is its output. In ordinary programming, you supply the input and the rules, and the computer produces the output.
This works beautifully when the rules can be stated exactly. Calculating an electricity bill from two meter readings, arranging a class list in alphabetical order, stopping a lift at the floor whose button was pressed — every one of these has a rule you can write down without argument.
Now try the same thing for a job like telling a ripe tomato from an unripe one, reading someone’s handwriting, or picking out a friend’s face in a group photograph. You can do all three easily. But you cannot say precisely how, and Ravi’s growing list of rules in the opener shows what happens when you try.
There is a second way to get a computer to do such a job. Instead of writing the rules, you collect a large number of solved examples — many tomatoes, each already marked ripe or unripe — and hand them to a procedure that works out for itself which measurements separate one group from the other. You supply the input and the correct output; the computer works out the rule. This is machine learning (ML).
The difference between the two ways is worth setting out side by side. Table 12.1 compares them.
The two ways are not rivals. Most real systems use both. A parking gate still uses a plain rule for opening the barrier, and machine learning only for reading the number plate. See Fig. 12.1-1.
Diagram
Fig. 12.1-1 – two flow blocks side by side. Left block labelled “Rules written by hand”: arrows from “Input data” and “Rules written by programmer” into a box labelled “Computer”, with one arrow out labelled “Output”. Right block labelled “Machine learning”: arrows from “Input data” and “Correct outputs (examples)” into a box labelled “Computer”, with one arrow out labelled “Rule worked out from the examples”
12.1.2 Machine Learning as a Part of Artificial Intelligence
You met artificial intelligence (AI) in Class IX. It is the wider field concerned with getting computers to carry out tasks that normally need human judgement — understanding speech, translating a sentence, planning a route, recognising an object.
Machine learning is one part of that field, and today it is the most widely used part. What marks it out is the source of its rules. In machine learning, the rules come from data instead of from a programmer.
Explained simply: a machine learning system improves at a task as it is given more examples of that task, without anybody rewriting its instructions.
Not every AI system uses machine learning. A program that plays noughts and crosses perfectly by following a fixed set of tactics is artificial intelligence, but there is no learning in it — its tactics were written by a person and never change. You will meet this distinction again as the rule based approach and the learning based approach.
Machine learning has one honest weakness, and it is worth knowing from the start. Because the rule is worked out from examples, the system can only be as good as those examples. Give it a thousand pictures of red tomatoes and none of yellow ones, and it will have no idea what to do with a yellow tomato. It will still produce an answer — a confident, wrong one.
12.1.3 Where Machine Learning Is Used Today
Machine learning suits any job where a great deal of past data exists and the pattern inside it is too fine for a person to write out. That describes a surprising number of everyday services.
Some common uses
- Health care: sorting X-ray and scan images so that unusual ones are shown to a doctor first, and flagging patients who may need closer watching.
- Banking and finance: spotting a card transaction that does not match a customer’s usual pattern and holding it for checking.
- Shops and online platforms: grouping customers by what they buy, and suggesting items similar to ones a customer has looked at.
- Vehicles that assist the driver: recognising lane markings, road signs and obstacles from camera images.
- Speech and language: turning spoken words into text, and translating between languages.
- Farming: sorting harvested produce by size and quality on a conveyor, and reading crop images to detect disease early.
In every one of these, notice what the system actually produces. It does not decide anything by itself; it computes an answer — a group, a number, or a label — and a person or another program acts on it.
12.2 Key Ideas in Machine Learning
Before you can follow how a machine learning system is built, you need names for its parts. There are only a handful of them, and each one answers a plain question: what does the learning, what comes out of it, what goes in, and what can go wrong.
12.2.1 Algorithms and Models
Suppose you hand a computer five thousand solved examples. Something has to read them and work out the pattern. That something is a procedure — a fixed sequence of steps, written by people, that the computer carries out. A procedure like this is called an algorithm.
You have already used algorithms without calling them that. A recipe is an algorithm for making poha. The steps for long division are an algorithm. What makes an algorithm useful is that it is exact: the same steps on the same starting values always give the same result.
Now, what does the algorithm produce when it has finished reading the examples? Not an answer — the examples already had their answers. It produces a rule: a set of numbers and comparisons that turn a fresh input into an output. This finished rule is called a model.
The model is the useful part. Once it exists, the examples can be put away. You feed the model a new cycle, a new photograph, a new reading, and it computes an output for it.
The two words are easy to mix up, so keep the order in mind: the algorithm is the process, the model is the product. Fig. 12.2-1 shows how they fit together.
Diagram
Fig. 12.2-1 – flow from left to right. A stack of labelled example rows feeds into a box marked “Algorithm”. An arrow from that box leads to a box marked “Model”. Below, a separate arrow shows “New, unseen input” entering the Model box, with an arrow out labelled “Output”
12.2.2 Training Data
The examples have to come from somewhere, and there have to be enough of them. A collection of data organised for a purpose is called a dataset. The part of a dataset used to build the model is called the training data, and the process of building the model from it is called training.
Table 12.2 shows what a small piece of training data looks like. The figures are a made-up set for a second-hand cycle shop, put together only to show the shape of a dataset.
Each row is one example. Reading down the table, you can already see a rough pattern: older cycles with more distance and more rust sold for less. An algorithm does the same thing, but with far more rows and far more care than the eye can manage.
Seven rows would never be enough in practice. A dataset this small lets a rule fit it perfectly by accident, in the way a straight line can always be drawn through any two points. Real training sets run to thousands of rows.
12.2.3 Features and Labels
Look at Table 12.2 again. The first three columns are things you can go and measure about a cycle. The last column is the thing you want the system to work out for a cycle whose price you do not know. These two kinds of column have their own names.
The measured input columns are the features.
The known output column is the label.
So in Table 12.2 the features are age, distance ridden and number of rust spots, and the label is the selling price. Choosing good features matters as much as collecting many rows; you will see why in Section 12.4.
Labels come in two kinds, and the kind decides what sort of job the system is doing.
- A categorical label names a group. An email is marked spam or not spam. A photograph is marked cat, dog or bird. There is no in-between value.
- A continuous label is a number that can take any value in a range. The price of a cycle, tomorrow’s temperature, the weight of a crate — all continuous.
12.2.4 Overfitting
Training a model is not simply a matter of making it agree with the training data. That is the trap.
Imagine a student who prepares for a class test by learning the answers to one practice sheet word for word — including a spelling slip in one of them. On that sheet the student would score full marks. Ask a slightly different question and the whole thing collapses, because nothing was understood, only copied.
A model can go the same way. If it is allowed to bend far enough, it will pass exactly through every training example, chasing not just the real pattern but also the small errors and odd cases in the data. It then agrees perfectly with the data it was built from and performs badly on anything new. This is called overfitting.
Fig. 12.2-2 shows the idea. Both curves pass near the same points, but only one of them is any use for a point that is not there.
Diagram
Fig. 12.2-2 – two small graphs side by side, each showing the same scattered dots. Left graph, labelled “Good fit”: a smooth line passing near the dots but not through all of them. Right graph, labelled “Overfitted”: a sharply wavy line bending to pass exactly through every dot
Overfitting is invisible if you only look at how well the model does on its training data — there it looks perfect. The only way to catch it is to test the model on examples it was never trained on, which is exactly what Section 12.4.2 is about.
12.3 Types of Machine Learning
Not every learning problem comes with the same kind of data. Sometimes you have the correct answers already. Sometimes you have plenty of data but no answers at all. And sometimes there is no data to start with, only a task that has to be attempted and scored.
Because of this, machine learning is divided into three main types: supervised learning, unsupervised learning and reinforcement learning. Fig. 12.3-1 sets them out.
Diagram
Fig. 12.3-1 – a tree with “Machine learning” at the top branching into three boxes: “Supervised learning (data with labels)”, “Unsupervised learning (data without labels)” and “Reinforcement learning (learning by trying and being scored)”. The supervised box branches further into “Classification” and “Regression”
12.3.1 Supervised Learning
Think of how a teacher helps you learn long division. You attempt a sum, the teacher tells you the correct answer, and you adjust the way you work until your answers match. You are being supervised: someone who knows the right answer is checking your work.
Supervised learning follows the same idea. The training data holds the correct output alongside every input, so the algorithm can compare what the model produces with what it should have produced, and keep adjusting until the two are close. This is supervised learning.
The training data for supervised learning is therefore a set of input–output pairs: the features are the input, the label is the output. Table 12.2 in the previous section was exactly this — three feature columns and one label column.
Almost everything you would think of as a prediction problem is supervised. Deciding whether a message is spam, reading a handwritten pin code, estimating tomorrow’s electricity demand — all of them start with a store of past cases whose answers are already known.
Which kind of supervised learning you are doing depends on the label. If the label names a group, the job is classification. If the label is a number, the job is regression.
12.3.1.1 Classification
Classification is used when the output has to be one of a fixed set of groups. There is no half-way answer: an email is either spam or it is not.
Common classification jobs include marking a message as spam or not spam, deciding whether a photograph contains a cat, a dog or a bird, judging a customer review as positive, negative or neutral, and separating ripe fruit from unripe fruit on a conveyor.
A classification model has one important limit. It can only produce a group it was trained on. Train it on cats, dogs and birds, then show it a photograph of a goat, and it will not report “something else” — it will pick whichever of the three it finds closest.
12.3.1.2 Regression
Regression is used when the output is a number rather than a group, and any value in a range is possible.
The second-hand cycle problem is regression: from age, distance ridden and rust spots, the model computes a price in rupees. Other examples are estimating a house price from its size and locality, forecasting the temperature for the next day, and predicting the yield of a field from rainfall and fertiliser used.
The difference between the two is only in the label, so the same data can pose either question. Predicting a patient’s exact temperature is regression; deciding whether a patient has a fever is classification.
12.3.2 Unsupervised Learning
Labels are expensive. Somebody has to look at each example and write down the correct answer, and for many datasets nobody ever has.
A grocery shop, for instance, may hold two years of billing records. Nothing in those records says what type of customer each buyer is — no one ever sat down and decided. Yet the records clearly contain structure. Some customers come every day for small amounts, some come once a month and fill a trolley. Finding that structure, without being told what to look for, is unsupervised learning.
The most common unsupervised task is clustering — putting similar examples into groups, where the groups themselves are decided by the data rather than by a person. The shop’s daily buyers and monthly buyers would emerge as two clusters without anyone naming them in advance.
Two other unsupervised tasks are worth knowing. One reduces the number of features, keeping only the ones that carry most of the information, so that later work is faster. The other looks for associations — items that keep appearing together in the same bill, for example.
Because there is no correct answer to compare against, unsupervised learning cannot be checked the way supervised learning can. The algorithm will always return groups. Whether those groups mean anything useful is for a person to judge.
12.3.3 Reinforcement Learning
Now think of a problem where neither labelled nor unlabelled data exists to start with. Nobody can hand a robot a table of correct leg movements for walking across a cluttered floor, because the correct movement depends on where the robot happens to be at that instant.
For such tasks the system has to gather its own experience. It tries something, sees what happens, and is scored. That is reinforcement learning.
The learner is called the agent, and everything it acts upon is called the environment. After each action the agent receives a reward — a number that is positive when the outcome was good and negative when it was bad. Over many attempts, the agent settles on the way of acting that collects the most reward over time. Fig. 12.3-2 shows the loop.
Diagram
Fig. 12.3-2 – a loop between two boxes. An arrow from the box “Agent” to the box “Environment” is labelled “Action”. Two arrows return from “Environment” to “Agent”, labelled “New state” and “Reward or penalty”
12.3.4 Telling the Three Types Apart
The quickest way to tell which type a problem needs is to ask what you have in hand. Do you have correct answers? Do you have data at all? Table 12.3 sets the three side by side.
12.4 The Role of Data in Machine Learning
In ordinary programming, the quality of the program depends on the programmer. In machine learning it depends on the data. The algorithm is fixed; what it produces is decided entirely by the examples it is given.
That is why most of the effort in a real machine learning project goes not into the clever part but into gathering, checking and preparing data.
12.4.1 Data and Information
Start with the two words themselves, because they are often used as though they meant the same thing.
Data is the raw material: plain recorded facts, with nothing said about what they mean. A column of numbers — 32, 35, 31, 36, 34 — is data. On its own it tells you nothing at all.
Information is what you get when data is given meaning. Say that those numbers are the highest temperature in degrees Celsius on five days in May at a school in Jaipur, and each number now says something. Say further that the average of the five is above the average for the same week last year, and you have information that could be acted on.
Machine learning sits between the two. It takes in data and produces something that yields information: a model whose outputs — a group, a number, a grouping of customers — mean something to the person who asked the question.
12.4.2 Training Data and Testing Data
Here is a fair question. Once a model has been trained, how do you find out whether it is any good?
The obvious answer is the wrong one. If you check the model against the very examples it was trained on, it will do well by construction, and an overfitted model will do best of all — it has effectively memorised those rows. Testing a model on its training data tells you nothing about how it will behave tomorrow.
So the dataset is split before training begins. One part is kept aside and never shown to the algorithm. This part is the testing data, and it is used only at the end, to measure how the model behaves on examples it has never seen. Fig. 12.4-1 shows a typical split.
Diagram
Fig. 12.4-1 – a long horizontal bar representing the whole dataset, divided into three unequal parts labelled “Training data (largest part)”, “Validation data” and “Testing data”. An arrow from the training part points to a box labelled “Algorithm builds the model”; an arrow from the testing part points to a box labelled “Model is measured”
The middle part in the figure is the validation data. It is used while the work is still going on, to compare one setting against another and to pick the better one, so that the testing data stays untouched until the very end.
The usual measure for a classification model is accuracy — the share of test examples on which the model’s output matched the true label:
\[\text{accuracy} = \frac{\text{number of correct outputs}}{\text{total number of test examples}} \times 100\]
Here the number of correct outputs is the count of test examples where the model’s output equalled the recorded label, and the total is the size of the test set. The answer is a percentage.
Table 12.4 is a made-up test set of ten messages for a spam filter, showing the true label of each and what the model produced.
Accuracy has an honest weakness worth stating. If only 5 messages in every 100 were spam, a lazy model that marked everything “not spam” would score 95% and catch nothing. A high accuracy figure means little until you know how the groups are balanced in the data.
12.4.3 Preparing and Cleaning the Data
Data collected from the real world is rarely fit to use as it stands. A school’s records may have a blank where somebody’s attendance was never entered, the same student’s name spelt two ways, one height recorded in centimetres and another in metres, and a stray reading of 200 kg for a Class X student.
None of this troubles a person reading the file. All of it troubles an algorithm, which has no way of telling an impossible value from a real one and will fit its rule around the mistake.
Cleaning the data means going through it and dealing with these problems. Missing values are either filled in sensibly or the row is dropped. Values that could not possibly be right are corrected or removed. Duplicate rows, which would otherwise count twice, are deleted.
Preparing the data goes one step further and puts everything into a form the algorithm can work with. Every column is brought to a single unit. Numbers on very different scales — a distance in thousands of kilometres beside a count of rust spots between 0 and 5 — are brought onto a common scale, so that the larger numbers do not swamp the smaller ones. Columns holding words rather than numbers are turned into numbers by an agreed coding.
12.4.4 Choosing the Right Features
Not every column in a dataset helps. Some carry the pattern you are after, some carry nothing, and a few actively mislead.
For the second-hand cycle prices in Table 12.2, age and distance ridden clearly matter. The colour of the paint almost certainly does not. Keeping a useless column costs time and gives the algorithm one more thing to latch on to by accident, which makes overfitting more likely. So the weakest columns are dropped, and the ones that carry most of the pattern are kept.
Sometimes a better feature can be built out of the ones already present. Age and distance ridden are more telling when combined: a five-year-old cycle that has done 6000 km has been ridden lightly, while a two-year-old one with the same reading has been worked hard. Dividing distance by age gives a new column, kilometres per year, that captures this in a single number.
Building and choosing columns in this way is a large part of what makes one machine learning project succeed where another, using the same algorithm, does not.
12.4.5 Biased Data and Unfair Results
A model can only describe the data it was shown. If that data leans one way, so will the model — and it will do so quietly, giving no sign that anything is wrong. A leaning of this kind in the data is called bias.
Suppose a fruit-grading system is trained only on produce from one district, where the mangoes happen to run small. Sent to a district where they grow larger, it will grade perfectly good fruit as oversized. Nothing in the model is broken. It was simply never shown the other kind.
The same thing happens with data about people, and there the consequences are heavier. A system trained on the past decisions of one office will reproduce whatever leanings those decisions carried, including ones nobody intended and nobody noticed.
12.5 Steps in Machine Learning
A machine learning project is not one act of cleverness. It is an ordered piece of work, and skipping a step early on shows up as a failure much later, when it is expensive to fix.
The whole process, from the first question to a system that keeps working month after month, is set out below.
Fig. 12.5-1 shows the same ten steps as a flow, with the two loops that make it a cycle rather than a straight line.
Diagram
Fig. 12.5-1 – a flowchart of ten boxes in order from “Define the problem” to “Monitor and maintain”. An arrow loops back from “Evaluate” to “Tune and improve” and on to “Train”; a second, longer arrow loops back from “Monitor and maintain” to “Collect the data”
12.5.1 From Problem to Working Model
Defining the problem
The first step is the one most often rushed. “Use AI in the school canteen” is not a problem statement. “Work out how many plates of a dish will be needed on a given day, from the day of the week and the past two years of sales” is one, because it says what goes in, what comes out and how you would know whether it worked.
Defining the problem also settles what kind of learning is needed. A number as output means regression; a group as output means classification.
Collecting the data
Next the examples are gathered. They may come from a database of past records, from sensors on a machine, from forms filled in by people, or from a public dataset published for the purpose. What matters is that they are relevant to the question and come from a source you can trust.
Preparing the data
The collected data is then cleaned and put into shape, exactly as described in Section 12.4.3. In most projects this takes longer than every other step put together.
Splitting the data
The prepared dataset is divided into training, validation and testing parts before any training begins. Doing the split first is what makes the final measurement honest.
Selecting the model type
Now an algorithm is chosen to suit the problem — one kind for predicting a number, another for sorting into groups, another for clustering. There is no single best choice, so more than one is often tried.
Training the model
The algorithm is run on the training data. It works through the examples, comparing the outputs it computes with the labels, and adjusting the numbers inside the model to make the difference smaller. When the difference stops improving, training stops and a model exists.
Evaluating the model
The model is then measured on data it has not seen, using a figure such as the accuracy you worked out in Section 12.4.2. For a model that predicts numbers, the usual measures are based on the size of the error — how far the computed value is from the true one, averaged over the test examples.
Tuning and improving
If the result is not good enough, the work goes back a step or two. The features may be changed, a different algorithm tried, or the settings adjusted. A setting that is fixed before training rather than learnt during it — how long training runs, for instance, or how large a step the algorithm takes each time — is called a hyperparameter. Tuning means searching for the combination of these settings that gives the best result on the validation data.
12.5.2 Watching the Model After It Is Deployed
A model that works on stored data has not yet done anything useful. Putting it where real inputs reach it — inside the canteen’s ordering screen, on the conveyor at the sorting shed, behind the bank’s transaction check — is called deployment.
Deployment is not the end of the project, and this is the step students most often leave out. The world the model was trained on keeps changing. Tastes change, prices change, a new variety of mango arrives, a new kind of fraudulent transaction appears. The model does not notice; it goes on applying the pattern it learnt, and its outputs slowly become less correct.
So the system’s performance is watched after deployment, using real outcomes as they become known. When the quality falls, fresh data is collected and the model is trained again. That is the long loop in Fig. 12.5-1, and it is why a deployed machine learning system needs looking after in a way that ordinary software does not.
12.5.3 The Steps in Action: Sorting Fruit
Now take the ten steps through one complete example. A farmers’ cooperative wants to grade mangoes on a conveyor into two grades — those fit for the premium crate and the rest — instead of having people do it by eye all day. Fig. 12.5-2 shows the arrangement.
Diagram
Fig. 12.5-2 – a conveyor belt carrying mangoes from left to right. Above the belt, a camera and a light. Beside the belt, a weighing section. At the far end, a small arm that pushes selected fruit into a second crate. Labels: “camera and light”, “weighing section”, “controller running the model”, “sorting arm”, “premium crate”, “regular crate”
Step 1 — Define the problem. For each mango, compute one of two grades, premium or regular, from measurements taken as it passes. Success means the machine agrees with the cooperative’s experienced grader on the great majority of fruit.
Step 2 — Collect the data. For several days, every mango is measured by the machine and then graded by the experienced grader, whose grade is recorded as the label. This gives a labelled dataset — so this is supervised learning, and because the label is a group, it is classification.
Step 3 — Prepare the data. Rows where the camera missed the fruit are removed. Masses recorded in kilograms are converted to grams so that one unit is used throughout. Mass and diameter are brought onto a common scale.
Step 4 — Split the data. The rows are divided into training, validation and testing parts, chosen so that fruit from every collection day appears in each part.
Step 5 — Select the model type. A classification algorithm is chosen, because the output is one of two groups.
Step 6 — Train the model. The algorithm runs on the training rows, adjusting the model until its grades agree as closely as possible with the grader’s labels. Table 12.5 shows the shape of those rows.
Notice the sixth row. It is a heavy, large mango that was still graded regular, because of its surface marks. A rule written by hand from mass alone would have got it wrong; that single row is the sort of case the training data exists to teach.
Step 7 — Evaluate the model. The trained model is run on the testing rows, and its grades are compared with the grader’s. The accuracy is worked out exactly as in Section 12.4.2.
Step 8 — Tune and improve. If too many premium mangoes are being sent to the regular crate, the features are reconsidered — perhaps a measure of colour should be added — and the model is trained again.
Step 9 — Deploy the model. The model is loaded onto the controller beside the belt. As each mango passes, the camera and the weighing section supply the features, the model computes a grade, and the controller signals the arm.
Step 10 — Monitor and maintain. The grader spot-checks crates each week. When a new variety comes in and disagreements rise, fresh labelled data is collected and the model is trained again.
12.6 Programming and Algorithms in Subjective Decision Making
A computer works with data and logic. Given the same inputs and the same instructions, it produces the same output every time, and it has no preferences of its own.
Yet the systems you use every day appear to have preferences. A video app suggests one film rather than another. An online shop puts a particular pair of shoes in front of you. A library system recommends a book you might enjoy. None of these has an agreed correct answer — they are matters of taste.
So how does a machine, which has no taste, produce an output that looks like taste? That is what this section is about.
12.6.1 Objective Decisions and Subjective Decisions
Start by separating two kinds of decision.
An objective decision has an agreed rule, and anyone applying that rule carefully reaches the same answer. Whether a number is greater than 30, whether a bill has been paid, whether a student has attended 75 per cent of classes — these are settled by checking, and there is nothing to argue about.
A subjective decision depends on the person making it. Which of two songs is better, which shirt suits you, which film to watch on a free evening — two sensible people will disagree, and neither is wrong.
Table 12.6 sets the two side by side.
Machines are built for the left-hand column. Human decision making covers both, and much of it sits in the right-hand column. The interesting question is what happens when we ask a machine to produce an output for a right-hand-column question.
12.6.2 How an Algorithm Handles a Preference
The trick is not to make the machine develop a taste. It is to turn preference into arithmetic.
A preference can usually be broken into a few things that matter, each of which can be measured. It can then be written as a score: multiply each measurement by a number saying how much it matters, and add up. The item with the highest score is put first. A number of this kind, saying how much one factor weighs against another, is called a weight.
Take a school library system that suggests a book to a student. It uses three factors: whether the book is in a genre the student has borrowed before, the average rating other students of that class gave it, and how recently books like it were borrowed. The library has set the weights at 5 for genre match, 2 for average rating and 3 for recency. Table 12.7 holds the figures for three books.
Look at what happened. Book B is not even in the student’s usual genre, but it did well on the other two factors and came first. The output looks like a judgement about taste. It is a sum.
The weights are where the preference actually lives, and they can be arrived at in two ways. A person can set them by hand, as the library did. Or they can be learnt: record which suggestions students actually borrowed, and let a machine learning algorithm adjust the weights until the scores line up with those choices. The second way is how large recommendation systems work.
12.6.3 What a Machine Is Really Doing
Once you have seen the sum, three further ideas become easy to place.
A rule of thumb that people use to decide quickly, without working everything out, is called a heuristic. Preferring what someone looked at this week over what they looked at last year is a heuristic. Such shortcuts can be written straight into an algorithm as an extra factor or a larger weight.
Subjective judgements are also full of ambiguity. Words like “good” and “bad” mean different things in different sentences — “not bad at all” is praise. Systems that work with written or spoken language, an area called natural language processing (NLP), handle this by weighing each word according to the words around it, and by working with likelihoods rather than certainties. A chatbot answering a customer’s complaint is doing exactly this: computing which reply best fits the pattern of the words it was given.
What none of this means is that the machine has developed judgement. The output of every system in this section is a number, computed from measurements and weights, and then compared with other numbers. The appearance of taste comes from the fact that the weights were fitted to the choices of many people.
12.7 Summary
- In ordinary programming a person writes the rules; in machine learning the computer works the rules out from examples. The second way suits jobs people do well but cannot state as a rule, such as recognising a face or judging a ripe fruit.
- Machine learning is a part of artificial intelligence. Not every artificial intelligence system learns — a system that follows tactics written by a person is artificial intelligence with no learning in it.
- An algorithm is a step-by-step procedure. A model is what the algorithm produces after reading the examples, and it is the model that computes an output for new data.
- Training data is the set of examples used to build the model. Each example holds the input values and the correct output.
- The measured input columns of an example are its features. The recorded correct output is its label. A categorical label names a group; a continuous label is a number.
- Overfitting is a model matching its training data too closely, learning the errors and odd cases along with the real pattern, so that it performs poorly on new data.
- There are three main types of machine learning: supervised learning, unsupervised learning and reinforcement learning.
- Supervised learning trains on labelled data. It is called classification when the output is a group, and regression when the output is a number.
- Unsupervised learning trains on data with no labels and finds the structure inside it. Clustering, which groups similar examples together, is its commonest task.
- In reinforcement learning an agent acts, receives a reward or a penalty from the environment, and adjusts its choices to collect more reward over time. It is used for robot navigation, gripping tasks and game playing.
- Data is raw recorded fact; information is data organised and given meaning.
- A dataset is split into training, validation and testing parts before training begins. The testing part is never shown to the algorithm, so that the final measurement is honest.
- Accuracy is the share of test examples whose computed output matched the true label. A high accuracy means little when one group is far more common than the other.
- Real data must be cleaned — missing values handled, wrong values corrected, duplicates removed — and prepared, with units made uniform and numbers brought onto a common scale.
- Choosing good features matters as much as collecting many rows, and a better feature can sometimes be built from two existing ones.
- Bias in the training data becomes bias in the model. A system trained on a narrow slice of data works badly for everything it was not shown, without giving any warning.
- The machine learning process runs in ten steps: define the problem, collect the data, prepare the data, split the data, select the model type, train, evaluate, tune and improve, deploy, and monitor and maintain.
- Deployment is not the end. The world keeps changing while the model does not, so performance is watched and the model is trained again on fresh data when it drifts.
- An objective decision follows an agreed rule and gives the same answer for everyone; a subjective decision depends on the person making it.
- A machine handles a subjective question by turning preference into arithmetic: each factor is given a weight, every option is scored, and the highest score is offered. The preference lies in the weights, set by a person or fitted to what many people chose.
- Heuristics can be built into an algorithm as extra factors, and natural language processing handles the ambiguity of words by weighing each word against the words around it.
- Machine learning systems are used on people as well as on things. When they are, their output should be treated as a suggestion for a person to review, and the training data should be checked for who is missing from it.
12.8 Key Terms
- Accuracy — the share of test examples on which the model’s output matched the true label, written as a percentage.
- Agent — in reinforcement learning, the learner that takes actions in an environment and receives rewards for them.
- Algorithm — a step-by-step procedure, made of a fixed set of instructions, for carrying out a task or solving a problem.
- Artificial intelligence (AI) — the field concerned with getting computers to carry out tasks that normally need human judgement.
- Bias — a leaning in the training data that the model reproduces, making its outputs unreliable or unfair for cases the data under-represented.
- Categorical label — a label that names one group from a fixed set of groups, such as spam or not spam.
- Classification — supervised learning in which the model computes a categorical output for a set of input features.
- Clustering — an unsupervised task in which similar examples are put into groups decided by the data rather than by a person.
- Continuous label — a label that is a number able to take any value in a range, such as a price or a temperature.
- Data — a collection of raw facts, values or readings recorded without interpretation.
- Dataset — a collection of data organised for a purpose, usually as rows of examples.
- Feature — one measurable property of an example, used as an input to the model.
- Heuristic — a rule of thumb used to decide quickly without working everything out.
- Hyperparameter — a setting fixed before training rather than learnt during it, such as how long training runs.
- Information — data that has been organised and given meaning so that it can be understood and used.
- Input — whatever is supplied to a program or a model for it to work on.
- Label — the correct output recorded for a training example, which the model is being taught to produce.
- Machine learning (ML) — the branch of artificial intelligence in which a computer works out the rules for a task from examples in data rather than being given those rules by a programmer.
- Model — the rule produced by a machine learning algorithm after it has processed the examples, used to compute an output for new, unseen data.
- Natural language processing (NLP) — the area of artificial intelligence concerned with working with written and spoken human language.
- Objective decision — a decision that follows an agreed rule, so that anyone applying the rule carefully reaches the same answer.
- Output — whatever a program or a model gives back after working on its input.
- Overfitting — a model matching its training data too closely, learning the errors and odd cases in that data along with the real pattern, so that it performs poorly on new data.
- Recommendation system — a system that scores available items for a user and offers the highest scoring ones.
- Regression — supervised learning in which the model computes a continuous numerical output for a set of input features.
- Reinforcement learning — machine learning in which an agent learns a task by acting, receiving a reward or penalty, and adjusting its choices to collect more reward over time.
- Reward — the number returned to an agent after an action, positive for a good outcome and negative for a bad one.
- Subjective decision — a decision that depends on the preference of the person making it, so that two reasonable people may disagree.
- Supervised learning — machine learning in which the model is trained on a labelled dataset, where every input carries its correct output.
- Testing data — the part of a dataset kept aside and never shown to the algorithm, used at the end to measure the model.
- Training data — the set of examples given to a machine learning algorithm, each holding the input values with the correct output for them.
- Unsupervised learning — machine learning in which the model is trained on data with no labels and works out the structure that lies within it.
- Validation data — the part of a dataset used during development to compare settings and choose between them, keeping the testing data untouched.
12.9 Objective Questions
Fill in the blanks. The measurable input properties of an example are called its ____________, while the correct output recorded for that example is its ____________.
State whether each statement is true or false.
- Unsupervised learning needs a labelled dataset.
- The testing data is never shown to the algorithm during training.
- A model stops needing attention once it has been deployed.
Choose the correct option. A model that agrees almost perfectly with its training data but performs poorly on new data is said to be
- deployed
- overfitted
- validated
- clustered
Choose the correct option. Working out the selling price of a second-hand cycle from its age and the distance it has been ridden is an example of
- classification
- clustering
- regression
- reinforcement learning
Fill in the blank. A setting that is fixed before training begins rather than learnt during it, such as how long training runs, is called a ____________.
Match each item in Column A with the correct description in Column B.
| Column A | Column B |
|---|---|
| a. Supervised learning | i. Groups similar examples when no labels exist |
| b. Unsupervised learning | ii. Measures a trained model on examples it has not seen |
| c. Reinforcement learning | iii. Trains on inputs paired with their correct outputs |
| d. Testing data | iv. Learns from a reward received after each action |
- Answer in one line. A spam filter is tested on 50 messages and its output matches the true label on 45 of them. State its accuracy as a percentage.
12.10 Concept Questions
Explain the difference between an algorithm and a model, using one example of your own.
Define overfitting. Explain why it cannot be detected by looking at the training data alone.
Differentiate between classification and regression. Give one example of each from everyday life.
A part of every dataset is kept aside before training begins. State what this part is called and give two reasons why it must never be used for training.
A library suggestion system scores a book using weights of 4 for genre match, 3 for average rating and 1 for recency. A book has a genre match of 1, an average rating of 4 and a recency value of 5. Work out its score, showing every step of your working.
A mango-grading machine is trained only on fruit from one district, where the mangoes grow small. It is then installed at a farm in a district where the same variety grows larger. Explain what is likely to happen to its grading, and why the machine gives no warning that anything is wrong.
12.11 Case-Based Questions
- A school canteen wants a system that works out how many plates of khichdi to prepare each day, so that less food is wasted. The record kept over one week is shown below.
| Day | Students present | Weather | Plates sold |
|---|---|---|---|
| Monday | 410 | Dry | 180 |
| Tuesday | 395 | Rainy | 210 |
| Wednesday | 400 | Dry | 175 |
| Thursday | 380 | Rainy | 205 |
| Friday | 415 | Dry | 190 |
| Saturday | 200 | Dry | 95 |
- Name the features and the label in this dataset.
- State whether this is a classification problem or a regression problem, and give a reason for your answer.
- The canteen has only one week of records. Explain one problem this creates and how the canteen could deal with it.
- A school library installs a system that suggests books to students. Its weights were fitted using two years of borrowing records collected entirely from Class X students. The library now switches the system on for Class VI as well, and within a month the Class VI teachers report that the suggestions are being ignored.
- Explain why the suggestions are likely to be poor for Class VI students, even though the system works well for Class X.
- Describe what the library should do to correct this, and say why adjusting the suggestions by hand afterwards would not be a proper fix.
- State one thing the library should tell students about how their suggestions are produced.