Chapter 8: Introduction to Artificial Intelligence

It is a little after seven in the morning. Ravi picks up the phone to check the road to school. He looks at the screen, the phone unlocks, and a map opens. The road he usually takes is shown in red, and the app offers him a different route.

While he waits at the gate, a message app finishes his words before he types them. His mother’s phone shows a set of photos it has quietly gathered together, all of them from last year’s school trip. In the kitchen, a speaker answers when his sister asks it to play a song.

Nobody at Ravi’s house wrote any of these programs. And nobody wrote a separate instruction for every road, every face, every photo and every song. Instead, each of these systems was built to work things out from examples it had been shown earlier. That is the idea behind artificial intelligence (AI) — computer systems built to do jobs that normally need human intelligence.

Four ordinary moments from this morning are marked in Fig. 8.0-1.

Diagram

Fig. 8.0-1 – a single morning scene in a home, with four labelled callouts: (1) a phone unlocking when it is held up to a face, (2) a map on a screen showing a crowded road and an alternative route, (3) a photo app grouping several pictures of the same trip, (4) a speaker responding to a spoken request for a song

NoteCuriosity Corner

Q1. What makes a machine “artificially intelligent”? Q2. Where did the name “artificial intelligence” come from? Q3. Can you tell whether you are chatting with a person or with a program? Q4. A farm, a bank and a hospital do very different work. What do they all use AI for? Q5. If an AI system treats one group of people unfairly, whose mistake is it?

This chapter will answer each of these five questions.

8.1 What Is Artificial Intelligence?

A calculator can multiply two large numbers faster than any person in your class. An ATM can count notes without making a mistake. A washing machine can run a full wash cycle while nobody watches it. All of these machines are useful, and all of them work the same way: somebody wrote down the exact steps, and the machine repeats those steps.

A set of steps written down in a fixed order, so that a machine or a person can follow it and reach a result, is called an algorithm. Every ordinary program is built from algorithms of this kind. The machine never has to work anything out for itself, because every case has already been thought about by the person who wrote the steps.

Where Fixed Steps Stop Working

Now try a different job. Look at the handwriting of two of your classmates. You can tell whose is whose in a second. But write down the exact steps that let you do it, and you will get stuck almost at once. How tall is a tall letter? How round is a round letter? Every rule you write has hundreds of exceptions.

The same problem comes up with speech, with photographs and with roads. There is no short list of rules that separates a cat from a dog in every picture ever taken, or that tells a phone which road will be crowded this evening. These jobs are not impossible — you do several of them every day — but they cannot be written as a fixed set of steps.

So computer scientists took another route. Instead of writing the rules, they wrote programs that are given a large number of solved examples and work out the pattern from those examples. The facts and examples given to such a program — pictures, recordings, numbers, past records — are called data. This is the idea at the centre of artificial intelligence, and it is what makes AI different from ordinary programming (see Fig. 8.1-1).

Diagram

Fig. 8.1-1 – two panels side by side. Left panel “Ordinary program”: a box labelled “rules written by a programmer” with an arrow from “input” through the box to “answer”. Right panel “AI system”: a box labelled “pattern worked out from data” fed by a stack of labelled examples, with an arrow from “new input” through the box to “answer”

NoteDefinition

(Artificial intelligence is the branch of computer science that builds computer systems able to carry out tasks that would normally need human intelligence — such as recognising pictures, working with everyday language or choosing an action in a new situation — by computing patterns from data rather than by being given the answer for every case in advance.)

Q. Is a calculator an example of AI?

A. No. A calculator follows fixed arithmetic steps written by a programmer, and it gives the same answer for the same input every time. Nothing in it is worked out from examples.

Treating every automatic machine as AI What goes wrong: Students often say that a lift, a traffic light on a timer or an automatic gate is AI, because it works without a person operating it. The fix: Working automatically is not the same as being AI. Ask a different question — was the machine given the rules, or was it given examples and left to compute the pattern? A traffic light that changes every sixty seconds is following a rule. A traffic light that adjusts its timing after processing camera data on how many vehicles are waiting is an AI system.

Q1. What makes a machine “artificially intelligent”?

A. Not speed, and not the fact that it runs by itself. A machine is called artificially intelligent when it handles a task that has no fixed set of rules, by computing patterns from data it has already been given, and then applying those patterns to something new.

What AI Still Cannot Do

AI systems can process far more data than any person, and they can keep doing it all day without getting tired. That does not make them better than people at everything. It is worth being clear, right at the start, about what they cannot do.

  • They do not produce genuinely new ideas. An AI system works from patterns in the data it was given, so it cannot step outside that data the way a person can when inventing something.
  • They do not feel emotions, and they handle human feelings poorly. A system may label a message as angry, but labelling a feeling is not sharing it.
  • They have no common sense and no morals. A person weighs what is fair or kind before acting. An AI system computes an output from its data, and nothing more.
  • They do not switch tasks easily. A system built to sort photographs is of no use for reading handwriting until it is built and trained again for that new job.

None of this is a small point. It is the reason AI systems are used as tools alongside people, and not as replacements for human judgement.

Take any two apps on a phone you are allowed to use. For each one, write down one job it does by following fixed rules, and one job it could only be doing by working out a pattern from data. Compare your list with a classmate’s.

8.2 A Brief History of Artificial Intelligence

The idea of a machine that could do the kind of work a mind does is much older than the computer. What changed in the middle of the twentieth century was that machines finally existed which could hold instructions and run them quickly. Once that happened, a natural question followed: how far can this go? The story of AI is really the story of people trying to answer that question, being disappointed, and then trying again with a better idea.

The main stages are shown in Fig. 8.2-1.

Diagram

Fig. 8.2-1 – a horizontal timeline running left to right with five labelled markers: “1956 – the name ‘artificial intelligence’ is used at the Dartmouth workshop”, “1960s–1970s – rule-based systems”, “1970s and late 1980s – the AI winters”, “1980s–1990s – artificial neural networks”, “2010s onwards – deep learning and large datasets”

The Name Is Chosen

In 1956 a summer workshop was held at Dartmouth College in the United States, bringing together a small group of researchers interested in machines that could carry out tasks needing intelligence. The name for the new field came from John McCarthy, who used the phrase “artificial intelligence” in the written proposal for that meeting. That is the moment the subject you are studying got its name.

Work from this period included the Logic Theorist, a program written by Allen Newell and Herbert A. Simon that could prove theorems in mathematics. Proving a theorem means finding a chain of correct steps from what is given to what must be shown. A program that could search for such a chain was strong evidence that machines could handle more than arithmetic.

Q2. Where did the name “artificial intelligence” come from?

A. From John McCarthy, who used the phrase in the proposal for a summer workshop held at Dartmouth College in 1956. That workshop is usually treated as the starting point of AI as a field of study.

Writing Down the Rules

Through the 1960s and 1970s, most AI work followed one approach: write the knowledge down as symbols and rules, and let the program apply those rules. A medical program of this type might hold hundreds of rules of the form “if this symptom and that test result, then consider this illness”. Because this style came first, it is often called good old-fashioned AI.

This approach worked well in narrow areas where an expert could actually list the rules. It failed everywhere else, for exactly the reason you saw in Section 8.1 — most useful tasks have far too many exceptions to write down.

Q. If rule-based systems worked in narrow areas, why did researchers move away from them?

A. Because every new task meant starting again, with experts writing thousands of fresh rules by hand. The system could not pick up anything from experience, so the effort never got smaller.

The AI Winters

Early researchers had promised a great deal, and progress was slower than promised. When results did not arrive, governments and companies cut their funding, and interest in the subject fell away. A period in which support and interest in AI drop sharply is called an AI winter. The first came in the 1970s, and a second followed in the late 1980s.

These were not wasted years. Quiet work continued, and much of it became the foundation of what came next. But they are a useful warning about a subject that still attracts large claims.

Learning from Data Instead

From the 1980s onwards, a different approach gained ground. Rather than writing rules, researchers built programs that adjust themselves as they process examples — the approach now called machine learning, in which a program improves its output on a task by computing patterns from data instead of being programmed with the answer.

One important tool for this is the artificial neural network, a program built from many simple units connected in layers, where each connection carries an adjustable number. As the network processes example after example, those numbers are adjusted so that its output moves closer to the correct answer. Geoffrey Hinton and Yann LeCun were among the researchers whose work made these networks practical.

Deep Learning and the Present Day

Networks with many layers stacked one after another are described as deep, and training such networks is called deep learning. The idea was not new, but two things arrived in the 2010s that it had always needed: very large collections of data, and computer hardware fast enough to process that data in a reasonable time.

The results changed the field. Programs became far better at recognising objects in photographs, at converting speech into text and at handling written language. Most of the AI you will meet in the rest of this chapter — in banks, on farms, in hospitals and on your phone — rests on this work.

In 1997 the chess program Deep Blue, built by IBM, won a six-game match against the world champion Garry Kasparov. Deep Blue did not learn from data in the modern sense; it searched enormous numbers of possible move sequences. The match is remembered because it was the first time a reigning world champion lost a full match to a machine.

8.3 The Turing Test

Suppose someone builds a machine and claims it is intelligent. How would you check? You could argue about what the word “intelligent” means, but that argument has been going on for centuries and has never been settled. A definition nobody agrees on is of no use to a person who has to give a straight answer about a particular machine.

The British mathematician Alan Turing faced this problem in 1950, in a paper on computing machines and thinking. His solution was to change the question. Instead of asking whether a machine can think, he asked something far more practical: can a machine hold a conversation so well that a person talking to it cannot tell it apart from another person?

This is a good question because it can actually be carried out. There is nothing to argue about at the end — either the person could tell, or the person could not. Turing called his arrangement the imitation game. Today it is known as the Turing Test.

NoteDefinition

(The Turing Test is a test in which a human judge holds a text conversation with a machine and with a person, and the machine is said to pass if the judge cannot reliably tell which of the two is the machine.)

Notice one detail in that description. The conversation is carried out in text only, so the judge never hears a voice or sees a face. Turing chose this on purpose. He was not interested in whether a machine could be made to look human; he was interested in what it could do with language.

Q. Why does the judge have to talk to a person as well, and not only to the machine?

A. Because there has to be something to compare against. Without a real person in the conversation, the judge would only be guessing about how a human reply usually sounds.

The next two sections look at how the test is actually run, and then at the important question of what a pass really tells you.

8.3.1 How the Turing Test Is Carried Out

The arrangement is deliberately simple, because anything complicated would give one side an unfair advantage. Three parties take part: a person who asks the questions, called the judge; a second person who answers them; and the machine, which also answers them. The judge is in one room, and the other two are out of sight and out of hearing. All three communicate only by typing (see Fig. 8.3-1).

Diagram

Fig. 8.3-1 – a judge seated at a keyboard and screen on the left; two text links labelled A and B running to the right into two separate closed rooms, one containing a person at a keyboard and the other containing a computer; a caption arrow noting that the judge does not know which room holds which

The judge is told the important part in advance: one of the two respondents is a machine and the other is a person. The judge’s job is to find out which is which, and is free to ask about anything at all — a story, a joke, the weather, a sum, an opinion about a piece of music.

running the Turing Test Input: one machine, one human respondent, one human judge, and two text-only links labelled A and B Output: a decision on whether the machine passed the test Steps:

1. Place the human respondent and the machine out of sight of the judge.
2. Connect both to the judge through text-only links, labelled A and B.
3. Tell the judge that one of A and B is a machine and the other is a person.
4. Let the judge type any questions to A and to B for an agreed length of time.
5. Have both respondents reply in text only.
6. At the end, ask the judge to state which of A and B was the machine.
7. Repeat the whole run with several different judges.
8. If the judges are no better at identifying the machine than they would be by
   guessing, the machine has passed.

Why it works: the judge can probe any subject at all, and no clue from appearance, voice or reaction time is available, so only the content of the replies can give the machine away.

Two details in those steps matter more than they look. The first is step 3. The judge is told a machine is present, so nobody is being tricked by surprise — the judge is on guard from the first message.

The second is step 7. One judge fooled once proves very little, because that judge may simply have asked easy questions. The test is only meaningful across several judges and several conversations.

Q. Would a machine give itself away by answering a long multiplication instantly and correctly?

A. It might, and Turing noticed this himself. He pointed out that a machine trying to imitate a person could wait before answering a hard sum, or even give a wrong answer. The test rewards convincing imitation, not correctness.

That last point tells you what the test is really measuring. A machine does not pass by being right about everything. It passes by producing replies that sit within the range of things a person might plausibly type — including hesitation, changing the subject and the occasional mistake.

8.3.2 What the Turing Test Does Not Prove

The Turing Test has been argued about for more than seventy years, and the argument is worth understanding, because it is really an argument about what we want from machines.

Start with what a pass does show. A machine that gets through several conversations with several alert judges is handling everyday language well. It is following the thread of a conversation, answering what was actually asked, and producing replies in the range a person would produce. That is a hard thing to do, and no program could do it by accident.

Now the limit. A pass says nothing about what is going on inside the machine. The system is computing likely replies from patterns in the language data it was built with. It does not know what a mango tastes like or what it means to miss a bus, and it is not aware of the conversation in the way you are aware of it. Passing the test shows that the imitation is good. It does not show that anything is being experienced.

There is a second limit, and it is more practical. A program can do well on the test by avoiding difficulty rather than by handling it — changing the subject, answering a question with a question, or claiming to be tired of a topic. A person doing this in conversation seems normal, so the judge learns nothing. The test can therefore reward evasion.

Q. Does that mean a system which fails the Turing Test is not really AI?

A. No. A system that reads scans in a hospital or picks out diseased plants in a field would fail at the first question, because it was never built to converse at all. It is still AI, and it is still useful.

That is worth holding on to. Conversation is one skill among many, and the test measures only that one. Most of the AI systems in the rest of this chapter would fail the Turing Test instantly, and it would not count against them.

Q3. Can you tell whether you are chatting with a person or with a program?

A. Often you cannot, at least not from a short exchange of typed messages — which is exactly the situation the Turing Test describes. But being unable to tell only means the imitation is good. It does not mean there is a mind on the other side computing anything like your own thoughts.

A system that people mistake for a person can lead them to share things they would not share with a company — health worries, money troubles, family details. Any service that answers customers with a chatbot should say so plainly at the start of the conversation, and should offer a way to reach a human being.

Why the Test Still Matters

For all its limits, Turing’s idea changed how people think about machines. Before it, the question “can a machine think?” had no way of being settled. Turing turned it into something that could be set up, run and judged.

The test also shaped what got built. The goal of holding a natural conversation drove decades of work on chatbots and voice assistants, and those systems are now used everywhere from bank helplines to school websites. The test is not a good measure of how useful an AI system is. It remains an excellent way of starting a discussion about what we actually mean when we call a machine intelligent.

8.4 Applications of AI in Different Fields

A bank, a wheat farm and a hospital have almost nothing in common. They employ different people, they own different equipment, and a good day means something different in each of them. Yet all three now use artificial intelligence, and often for surprisingly similar reasons.

The reason is that each of them has ended up holding a great deal of data. A bank records every transaction. A farm can be photographed from a drone or a satellite every few days. A hospital stores scans, test results and case histories. Once data of that size exists, no team of people can read all of it, and the jobs that matter most in each field turn out to be jobs with too many exceptions to write down as rules.

The Same Three Jobs, Again and Again

Look closely at the AI systems used across all these fields and you will keep meeting the same three shapes of job.

  1. Recognition — finding something in a picture, a recording or a record. A diseased leaf, a defective part on a production line, a face at a gate.
  2. Prediction — computing what is likely to happen next from what has happened before. Which machine will need repair, which road will be crowded, how much stock a shop will sell.
  3. Handling routine work — dealing with the large number of ordinary, repeated tasks so that people are left with the difficult ones. Answering common questions, sorting forms, checking simple conditions.

Fig. 8.4-1 shows how the same three jobs appear in three very different places.

Diagram

Fig. 8.4-1 – three columns headed “Farm”, “Bank” and “Hospital”. Each column shows its own data source at the top (drone photographs of a field; transaction records; scans and test results), all three feeding downwards into the same three labelled boxes: “Recognition”, “Prediction”, “Handling routine work”

Q4. A farm, a bank and a hospital do very different work. What do they all use AI for?

A. For the same three things. Recognising something in an image or a record, predicting what is likely to happen next from past data, and taking over routine repeated work. The data is completely different in each place; the shape of the job is not.

How This Part of the Chapter Is Arranged

The sections that follow take a tour through the fields where AI is now in regular use: commercial work, industry, medical/health care, defense, banking, entertainment, transport, security and agriculture.

As you read, keep two questions in mind for every example. What data does this system need before it can work at all? And what happens when it gets something wrong? A system that suggests a film has a very different consequence from one that flags a bank customer as suspicious, even though the computing behind them may look much the same.

8.4.1 AI in Commercial Work and in Industry

Commercial work means buying and selling — shops, online stores, wholesalers and the people who supply them. Industry means making things — factories, workshops and plants. The two are neighbours, because everything sold in the first was made in the second.

In Commercial Work

Think about the owner of a small bakery. Every evening she has to decide how much to bake for tomorrow. Bake too little and customers leave without buying. Bake too much and the extra is wasted. She decides from memory and from a feel for the day of the week.

A large store faces the same decision, but for thousands of items across hundreds of branches, and nobody has a feel for all of that. What the store does have is a record of every sale it has ever made. An AI system built for demand forecasting — computing how much of an item is likely to sell in a coming period — processes that record and produces a number for each item in each branch.

The simplest version of this idea needs nothing more than an average, and you can work it out yourself. Table 8.1 holds one week of bread sales at a small bakery.

Table 8.1 – bread sold at a bakery on five working days | Day | Loaves sold | |—|—| | Monday | 42 | | Tuesday | 38 | | Wednesday | 45 | | Thursday | 40 | | Friday | 50 |

a first estimate of tomorrow’s bread order

Given: loaves sold on five days — 42, 38, 45, 40 and 50

Find: the average number of loaves sold in a day

Working: total sold = 42 + 38 + 45 + 40 + 50 = 215 loaves

average = 215 ÷ 5 = 43

43 loaves per day

That average is an honest starting point, and it is also a good example of how far a simple method takes you. It treats Friday and Monday as though they were the same day, and it knows nothing about a festival, a wedding season or a week of heavy rain. A real forecasting system processes all of those as extra columns of data, which is precisely why it is built with AI rather than with a single sum.

The same field record supports several other uses:

  • Recommendation systems compute which products a customer is likely to want, from what similar customers bought and what this customer looked at before.
  • Chatbots answer the ordinary questions — where is my order, how do I return this — leaving the difficult complaints to staff.
  • Fraud checks compare a card payment against the pattern of that customer’s earlier payments and hold back anything that sits far outside it.
  • Some stores have removed the checkout counter altogether. In Amazon Go stores, cameras and shelf sensors record what a customer picks up, and the bill is prepared without anyone scanning items.

Q. A recommendation system suggests a book you have already read. Has it failed?

A. Not really. It computes what is likely, not what is certain, so some suggestions will always miss. It has failed only if most of its suggestions are useless to you.

Some online sellers adjust prices for different customers, using what they know about each one. A customer who has searched for an item many times may be shown a higher price than a first-time visitor. Two people paying different amounts for the same item, without being told why, is hard to defend. A fair rule is that a price change should be something the seller is willing to explain openly.

In Industry

Now step into a factory. Two problems there have troubled managers for as long as factories have existed: catching faulty pieces before they reach a customer, and stopping a machine from breaking down in the middle of a shift.

Checking every piece by eye is slow, and a tired inspector at the end of a shift will miss more than a fresh one at the start. A camera does not get tired. An AI system trained on photographs of good pieces and faulty ones computes, for each new piece, whether it matches the faulty pattern (see Fig. 8.4-2).

Diagram

Fig. 8.4-2 – photograph of a camera-based inspection station above a factory conveyor belt, with parts passing beneath the camera and a screen alongside showing the captured image

camera-based quality inspection What it is: a fixed camera above a production line, connected to a computer that decides whether each piece is acceptable. How it works: the system is first given a large set of photographs of pieces already marked as good or faulty. It computes the visual patterns that separate the two, and then applies those patterns to each new photograph as the piece passes. Used for: checking printed circuit boards, bottle caps, tablets in a blister pack and welds on a metal frame. Watch out: it only recognises the kinds of fault it was given examples of. A new fault, never seen in the training photographs, can pass straight through.

The second problem is handled by predictive maintenance, in which sensor data from a machine is used to compute when it is likely to need repair. Vibration, temperature and current drawn all change slowly as a bearing wears. A person cannot watch those readings for a hundred machines at once; a program can, and it can call for a repair during a planned break instead of in the middle of production.

Factories use AI in several other ways. It schedules deliveries of raw material so that stock does not run out or pile up. It monitors electricity use across a plant and points to where it is being wasted. And it watches safety cameras for people entering areas they should not be in.

One development worth knowing about is the cobot, short for collaborative robot — a robot built to work in the same space as people rather than behind a safety cage. Cobots take the repetitive part of a job, such as lifting and holding, while a worker does the part that needs judgement.

A cobot is built to move slowly and to stop when it meets resistance, but it is still a powered machine with real force behind it. Never reach into a robot’s working area while it is powered, and always disconnect the power supply before adjusting, wiring or cleaning any part of it.

8.4.2 AI in Medical/Health Care and in Defense

These two fields sit oddly together in a list, but they share one feature that shapes how AI is used in both: a mistake is expensive, and sometimes it costs a life. That single fact changes how these systems are built and how much authority they are given.

In Medical and Health Care

A radiologist reading chest X-rays may look at a great many images in a day. Almost all of them are normal. The few that are not may differ from a normal image only in a small, faint region, and the longer the day goes on the easier that region is to miss.

This is a recognition job of exactly the kind described in Section 8.4. A system given a large number of scans, each already marked by doctors as normal or showing a particular condition, computes the visual patterns that separate them. On a new scan it marks the regions that match a pattern of concern and passes the image to the doctor with those regions highlighted (see Fig. 8.4-3).

Diagram

Fig. 8.4-3 – a chest X-ray displayed in review software, with a coloured rectangle drawn around one region that the system has marked for a doctor to examine, and a plain side panel listing the marked regions

Notice who makes the final call. The system does not give a diagnosis. It computes which regions deserve a closer look, which means the doctor spends attention where it is most likely to be needed. The diagnosis, and the responsibility for it, stay with the doctor.

an AI system that flags regions in a medical scan What it is: software that examines a scan and marks regions matching patterns linked to a particular condition. How it works: it is built from a large collection of scans already labelled by doctors. From these it computes the visual patterns associated with each label, and it applies those patterns to each new scan, marking regions that match. Used for: screening chest X-rays, examining photographs of the retina for eye damage caused by diabetes, and sorting a day’s scans so that the most urgent are read first. Watch out: if the scans it was built from came mostly from one type of machine or one group of patients, it can perform noticeably worse on scans from elsewhere.

A second use has nothing to do with images. A patient sent home after treatment used to be checked only at the next appointment, which might be weeks away. Remote patient monitoring — the continuous checking of a patient’s condition at home using connected devices — closes that gap. A wearable device records heart rate and other readings, software compares them against the patient’s own usual range, and anything unusual raises an alert for the medical team to look at.

The same approach, applied to whole populations rather than one patient, helps public health teams track a contagious disease. Records of reported cases, tests and movement of people are processed together to compute which districts are likely to see cases rise, so that staff and supplies can be sent before the rise happens rather than after.

Q. If the software is good at spotting a condition, why not let it give the diagnosis?

A. Because it computes a match against past examples and nothing more. It does not examine the patient, ask about symptoms, or weigh what a wrong answer would cost this particular person. Those are the parts of a diagnosis that a doctor does.

Medical data is among the most private information a person has. Any system built from patient scans or records must use data that patients agreed to share, with names and identifying details removed, and it must be checked on patients of different ages, backgrounds and body types before it is used in a hospital — otherwise it may work well for some groups and badly for others.

In Defense

Armed forces have always had more information arriving than people can read: photographs, radio traffic, sensor readings, maps, weather. AI is used mainly to sort that flood into something a commander can act on.

An unmanned aerial vehicle (UAV) is an aircraft flown without a pilot on board. AI systems hold it steady on a planned route, keep it clear of obstacles, and process the camera images it sends back, marking vehicles or movement for a human operator to examine (see Fig. 8.4-4). The same idea covers patrolling long borders and searching areas that would be dangerous to enter.

Diagram

Fig. 8.4-4 – an unmanned aerial vehicle flying over ground, with a camera feed shown as an arrow to a ground station screen; on the screen, two moving objects are outlined and labelled “marked for operator review”

Other uses follow the same pattern of sorting and predicting:

  • Cybersecurity — computing which activity on a military network sits far outside its normal pattern, so that an intrusion is caught early.
  • Planning and decision support — processing supply, movement and weather data to compare possible plans before one is chosen.
  • Simulated training — training systems that place personnel in realistic situations without the cost or risk of a live exercise.
  • Language work — sorting and translating large volumes of text so that analysts read the small part that matters.

Some weapon systems are built to select and attack a target with no person involved at the moment of the decision. Many countries, researchers and international bodies argue that a machine computing a pattern match must never be the last step before a life is taken, and that a human being should always carry responsibility for such a decision. This debate is unsettled, and it is one of the most serious open questions in the whole of AI.

8.4.3 AI in Banking and in Entertainment

A bank and a streaming service look like opposites. One is careful and quiet, the other is loud and full of pictures. Both, though, are trying to pick a small number of items out of an enormous list — the bank picking the few payments worth stopping, the streaming service picking the few titles worth showing you.

In Banking

Start with a problem every bank has. Millions of card payments pass through it in a day, and a small number of them are being made by someone who stole the card. A person cannot check them, and a fixed rule cannot either. “Block every payment above ten thousand rupees” would stop a great many honest purchases and miss a thief buying something cheap.

What actually gives a thief away is a break in pattern. Your own payments have a shape: certain kinds of shops, certain times of day, a certain part of the country, a familiar phone or card machine. A payment that departs from that shape in several ways at once is worth a second look. This is a job for AI because the shape of your spending is not something anyone could write down as a rule — it has to be computed from your own past transactions.

checking a card payment before it is approved Input: the details of a new payment, and the customer’s record of past payments Output: a decision to approve the payment or hold it for verification Steps:

1. Collect the features of the new payment: amount, time, place, type of
   shop, and the device or machine used.
2. Compare these features with the pattern computed from this customer's
   past payments, and with patterns from payments already known to be
   fraudulent.
3. Compute a risk score for the payment.
4. If the risk score is below the bank's threshold, approve the payment.
5. If it is above the threshold, hold the payment and ask the customer to
   confirm it, by a message or a call.
6. Record what the customer answers, and add that outcome to the data
   used for future checks.

Why it works: a stolen card is usually used in a way that differs from the real customer’s habits in several small ways at the same time, and a risk score computed from many features together picks up that combination.

Diagram

Fig. 8.4-5 – a flow chart of a card payment: “payment made” leading to a box “risk score computed from payment features and past pattern”, then a decision diamond “score above threshold?” with a “no” branch to “payment approved” and a “yes” branch to “hold and ask customer to confirm”, and a return arrow from the customer’s answer back into “past pattern”

Q. Why does a bank sometimes block a perfectly genuine payment?

A. Because a genuine payment can still break the pattern — a first purchase in a new city, or an unusually large amount. The bank sets its threshold knowing that a wrongly held payment is annoying, while an approved theft is far worse.

The same machinery, pointed at different data, does several other jobs in a bank:

  • Credit scoring — computing how likely a person applying for a loan is to repay it, from their record of earlier borrowing and repayment. The result guides the bank’s decision on the loan.
  • Watching for money laundering, by finding chains of transfers between accounts that together form a suspicious shape, even when no single transfer looks odd.
  • Answering routine customer questions through a chatbot at any hour — balance, statement, how to block a lost card.
  • Algorithmic trading — programs that buy and sell shares automatically, computing from market data far faster than a person could act.

Some lenders now compute a credit score using data far beyond a person’s repayment record, such as their phone use or online activity. A person can then be refused a loan for reasons they were never told and cannot argue against. Anyone refused should be given the reason in plain language and a way to have the decision looked at by a human being.

In Entertainment

A streaming service may hold more titles than you could watch in a lifetime. The old solution was a shelf marked “new releases” and another marked “popular”, the same for every customer walking in. That works badly when the catalogue is enormous and the customers want very different things.

So these services rank the catalogue separately for every viewer, from what that viewer watched, how long they stayed with it, what they abandoned after five minutes, and what people with similar viewing chose next. What you see on the home screen is that ranking.

AI does quieter work on these platforms too. It generates subtitles by converting speech in a film to text. It groups a catalogue by mood or theme without anyone tagging each title by hand. And in video games it controls opposing characters, adjusting their behaviour as the player improves so that the game stays challenging.

There is an honest limitation here, and it is worth thinking about. A system built to show you more of what you already liked will keep narrowing what it offers. If you watch three films of one kind, you will be shown a fourth, and the range you are offered quietly shrinks. The way out is simple, and entirely in your hands: search for something outside your usual range now and then, and the ranking will widen again.

8.4.4 AI in Transport and in Security

Transport and security both deal with the same awkward fact: the situation changes every second, and a decision taken a minute too late is worth nothing. Systems in these fields are built to process data as it arrives rather than at the end of the day.

In Transport

Stand at a busy junction in the evening and watch the signal. One arm of the junction is empty while its light is green; another has vehicles backed up for a hundred metres and its light is red. The signal is running on a timer that was set once and never changed, and the timer knows nothing about this evening.

An adaptive signal replaces the timer with data. Cameras or sensors at each arm count the vehicles waiting, and a program computes how long each arm’s green should last for this cycle (see Fig. 8.4-6). The counting is a recognition job — separating vehicles from shadows, road markings and people — which is why it is done with AI and not with a simple sensor.

Diagram

Fig. 8.4-6 – a four-arm junction seen from above, with a camera at each arm feeding a box labelled “vehicles counted on each arm”; an arrow from that box to “green time computed for this cycle”, and arrows from there back to the four signal posts

Route planning in a map app works on the same principle at a larger scale. Position data from many vehicles shows where traffic is moving slowly right now, and past records show what this road is usually like at this hour on this day. From both together, the app computes an estimated travel time for each possible route and offers the shortest.

The same computing supports other parts of transport:

  • Ride-hailing services such as Uber and Ola match a waiting passenger to a nearby driver, and compute fares that rise when many people are booking at once.
  • Delivery companies compute the order of stops for a van so that a full day’s parcels are covered in the least distance.
  • Cars carry driver-assistance features that keep the vehicle within its lane, hold a safe gap from the vehicle ahead, and apply the brakes when something appears in front.

Treating driver assistance as a self-driving car What goes wrong: because a car can hold its lane and brake by itself, a driver assumes the car is driving and stops paying attention to the road. The fix: these features are built to assist a driver who remains fully in charge. They compute an action from camera and radar data, and they fail in exactly the conditions that make driving hard — heavy rain, poor light, faded lane markings, an unusual obstacle. The driver’s hands and attention stay with the vehicle.

Q. A map app says a road is clear, but you arrive and find it jammed. What went wrong?

A. Probably nothing in the program. It computed from data collected a few minutes earlier, and a lorry breaking down after that is not in any of its data. Predictions are about what is likely, not what is certain.

In Security

Security work has always been about noticing the one thing that does not belong — one message among thousands, one face among hundreds, one pattern of activity on a network that nobody authorised. That is a recognition job of the hardest kind, because the ordinary case outnumbers the unusual one by an enormous margin.

Consider email. A phishing message is one pretending to come from a bank, a school or a company, written to make the reader hand over a password. Attackers change the wording constantly, so a list of banned phrases stops working within days. A filter built with AI instead computes patterns across many features at once — how the sender’s address relates to the name shown, where the links actually lead, how the wording compares with genuine mail from that organisation. Similar systems examine how a program behaves on a computer, holding back software that behaves like malware even when that exact program has never been seen before.

Identity checks are the second large use. Instead of a password that can be shared or stolen, a system checks something about the person themselves.

face recognition at an entry gate What it is: a camera and computer at a door or gate that check whether a face matches one already registered. How it works: the system computes a set of numbers describing the geometry of the face in the camera image — distances and proportions between features — and compares them with the numbers stored for registered faces. If the two sets are close enough, the gate opens. Used for: staff entry at offices, boarding checks at airports, and unlocking a personal phone. Watch out: accuracy falls with poor light, a partly covered face or an unusual camera angle, and systems built mostly from photographs of one group of people have been shown to work less well on others.

The third use is watching public spaces, and it is the one that needs the most care. Camera systems can pick out a face in a crowd, follow a vehicle across a city, or mark behaviour that departs from the usual pattern at a station. Some police forces also use software that computes where crimes are more likely to occur, from records of past crimes, and sends patrols accordingly.

A system that computes from past crime records will send patrols back to the areas that were policed most heavily before, which produces more recorded crime there and confirms its own prediction. The people living in those areas are watched more closely for reasons that have nothing to do with what they have done. Wide camera surveillance raises the same difficulty: people who have done nothing are recorded, and the record can be misused later. Responsible use means telling people plainly where such systems operate, keeping recordings only as long as they are needed, and never treating a match by software as proof on its own.

8.4.5 AI in Agriculture

A farmer walking a field can see it properly only a few rows at a time. A disease that has just started in one corner, a patch where water is not reaching, a stretch where the soil is short of one nutrient — all of these are visible, but only if somebody happens to walk past that spot at the right time. On a large holding, nobody does.

This is the gap AI fills in farming. A camera carried by a drone or a satellite covers the whole field in one pass, and software processes those images to compute where the crop is not doing well (see Fig. 8.4-7). What the farmer receives is not a photograph to study but a map of the places worth visiting.

Diagram

Fig. 8.4-7 – photograph of a small drone flying low over a green crop field, with the camera mounted beneath it clearly visible

Finding Disease Early

Catching a disease in its first week rather than its third can decide the harvest. Farmers now use phone apps built for this: the farmer photographs an affected leaf, and the app computes which known disease pattern the photograph matches most closely, then names it along with the usual treatment.

identifying a crop disease from a leaf photograph What it is: a phone app that names the likely disease affecting a plant from a photograph of its leaf. How it works: the app is built from a large collection of leaf photographs, each already labelled by agricultural scientists with the disease shown. It computes the visual patterns — colour, spot shape, spread along the veins — that separate one disease from another, and matches a new photograph against those patterns. Used for: early identification of common diseases and pest damage in crops such as wheat, rice, cotton and tomato. Watch out: it can only name diseases it has examples of, and a poorly lit or blurred photograph can produce a confident-looking match that is wrong. The result is a starting point for a farmer or an agricultural officer, not a final verdict.

Q. Why can the app not simply be given a written description of each disease instead of thousands of photographs?

A. Because nobody can describe in words exactly how brown a spot must be, or how ragged its edge, to belong to one disease rather than another. That boundary has to be computed from examples.

Spending Less by Working in Patches

Weeds do not grow evenly. They come in patches, and the rest of the field has none. Yet a sprayer crossing the field treats every square metre the same way, because the person driving it cannot stop and start for each patch.

A sprayer with a camera and AI can. It recognises weeds against the crop as it moves, and opens only the nozzles above a patch. The saving is easy to work out.

chemical saved by spraying only the weed patches

Given: a field of area 2,000 m²; weed patches covering 300 m² in total; a spray applied at the rate of 1 litre for every 100 m² treated

Find: the quantity of spray saved, and the saving as a percentage

Working:

spray needed for the whole field = 2,000 ÷ 100 = 20 litres

spray needed for the patches only = 300 ÷ 100 = 3 litres

saving = 20 − 3 = 17 litres

A percentage is simply a part out of a hundred, so the saving as a percentage is the saving divided by the original quantity, multiplied by 100:

percentage saved = (17 ÷ 20) × 100 = 85

17 litres saved, which is 85% of the chemical that would otherwise have been used

That saving is money for the farmer, and it is also less chemical entering the soil and the groundwater. Fig. 8.4-8 shows the two approaches side by side.

Diagram

Fig. 8.4-8 – two plans of the same rectangular field. The left plan is shaded uniformly and labelled “whole field sprayed – 20 litres”; the right plan is shaded only in three small irregular patches and labelled “patches sprayed – 3 litres”

Other Uses on the Farm
  • Precision irrigation — soil moisture readings and weather forecasts are processed together to compute how much water each part of a field needs, so that water is not wasted on ground that is already wet.
  • Yield prediction — past yields, rainfall records and current crop images are used to estimate how much a field will produce, which helps a farmer plan storage and sale.
  • Autonomous equipment — tractors guided by satellite positioning that sow and harvest in straight, evenly spaced passes without a driver steering.
  • Livestock monitoring — collars and cameras that record how much an animal moves, eats and rests, raising an alert when the pattern changes in a way that often means illness.
  • Sorting and grading — cameras that separate harvested produce by size, colour and damage as it moves along a belt, far faster than hand sorting.

None of this arrives free. A drone, a sensor network or a guided tractor costs money that a smallholder may not have, and most of these systems need a working phone network in the village. That is why the tools spreading fastest in India are the cheapest ones — a phone app and a camera that a farmer already owns.

Photograph a healthy leaf and a damaged leaf from a plant in your school garden. Write down every visual difference you can see between them, then decide which of those differences a program could compute from the photograph alone.

8.5 Commonly Used AI Applications in Daily Life

The fields in Section 8.4 were mostly places you visit — a bank, a hospital, a factory. This section is about the AI you meet without going anywhere, on an ordinary day, usually without noticing it at all.

Online Shopping

The moment you open a shopping app it has already arranged itself around you: what appears first, what is grouped under “you may also like”, which size it assumes, which sellers it shows. Search for a school bag and the results are ordered not only by how well they match your words but by what people who typed similar words went on to buy.

Search Engines

A search engine has to pick a handful of useful pages out of billions. It computes a ranking for every page against your words, using what the page contains, which other pages link to it and what people who searched similarly found useful. It also repairs your query along the way — correcting a spelling, expanding a short form, treating “cheap” and “low cost” as the same request.

Chatbots

Type a question on a bank’s website, a railway booking page or a school portal, and the first reply usually comes from a program. It computes which of the questions it was built for your message is closest to, and returns the answer for that one. Well-built chatbots hand you over to a person as soon as your question sits outside their range.

Voice Assistants

A voice assistant is a program that takes a spoken instruction and carries out an action. Speaking is far quicker than typing, and it leaves your hands free — which is why these appear in phones, speakers and cars.

Getting from a sound to an action takes several separate steps (see Fig. 8.5-1).

Diagram

Fig. 8.5-1 – four boxes joined by arrows, left to right: “microphone records sound”, “speech converted to text”, “text matched to a known request”, “action carried out”; a small side arrow from the last box back to a speaker icon labelled “spoken reply”

a voice assistant What it is: a program in a phone, speaker or car that responds to spoken instructions. How it works: the microphone records the sound, and a system built from a great many recordings paired with their written text converts your speech into words. A second stage matches those words to one of the actions the assistant can perform, along with any detail it needs — a song name, a city, a time. The action is then carried out and a reply is spoken back. Used for: playing music, setting alarms and reminders, asking for the weather, and controlling lights or fans in a connected home. Watch out: accuracy drops with background noise, with accents the system saw little of in its recordings, and with names of local places and people. Anything it cannot match to a known action, it will either refuse or guess at.

Entertainment Portals

Streaming apps for films, shows and music arrange their catalogues for each viewer, and they also do quieter work in the background — generating subtitles, and grouping titles by mood or theme without anyone labelling each one by hand.

Facial Recognition

Facial recognition is the computing of a match between a face in a camera image and a face already registered. You meet it every time you unlock a phone by looking at it, and travellers meet it at several Indian airports, where a registered face can be used instead of showing documents at each checkpoint.

Driver Assisting Vehicles

Many cars now carry features that read the road through cameras and radar: staying centred in a lane, holding a safe gap from the vehicle ahead, warning about a vehicle in the blind spot, and braking when something appears in front. These assist a driver who stays fully in charge of the vehicle.

Augmented and Virtual Reality

Augmented reality (AR) places computer-generated objects into a view of the real world, usually through a phone camera. To do this the software must first work out, from the camera image, where the floor is, where the walls are and how the phone is moving — which is a recognition job. Furniture apps that show a sofa standing in your own room, and camera filters that follow a face as it turns, both work this way.

Virtual reality (VR) replaces the real view entirely with a generated one, seen through a headset. AI is used there to track head and hand movement quickly enough that the view keeps up with you, and to control the characters inside the generated world.

Table 8.2 collects the everyday examples from this section in one place.

Table 8.2 – everyday fields and some commonly used AI applications in each | Field | Commonly used applications | |—|—| | Online shopping | Amazon, Flipkart, Myntra | | Search engines | Google Search, Bing, DuckDuckGo | | Chatbots | customer-service bots on bank, railway and school websites | | Voice assistants | Amazon Alexa, Apple Siri, Google Assistant, Samsung Bixby | | Entertainment portals | YouTube, Netflix, Spotify, Amazon Prime Video | | Facial recognition | phone face unlock, DigiYatra at Indian airports | | Driver assisting vehicles | Tesla Autopilot, Honda Sensing, BMW Driving Assistant | | Augmented/virtual reality | Google ARCore, Apple ARKit, Microsoft HoloLens |

Q. Does an AI system have to be running on the phone in your hand?

A. Usually not. Most of these send your request over the internet to a large computer elsewhere, which does the computing and sends back the result. That is why several of them work poorly with a weak connection.

Nearly every application in this section works better the more it knows about you, so nearly every one of them is collecting something — what you searched, where you were, what you said to the speaker in your room. Before using any of these, it is worth opening the app’s settings once and looking at what it is permitted to collect, what it keeps and for how long. Most allow far more to be switched off than people expect.

8.6 Benefits of Using AI

Section 8.4 asked where AI is used. This section asks a sharper question: what is actually gained by using it? A benefit is never a benefit in the abstract. It is always a gain compared with how the same work was done before — by hand, by a fixed rule, or not at all because nobody had the time.

Ten such gains are listed below. As you read them, notice how often the gain is not that a machine does the job better than a person could, but that it does the job at a size or a speed no group of people could manage.

  1. Automation. A task that once needed a person at every step can be carried through from start to finish by a system. In a district office, a form arriving by post used to be opened, read, sorted into a category and passed to the right desk. A system that reads the scanned form and routes it does the whole chain without a person touching it.

  2. Smart decision making. Decisions improve when they rest on all the evidence rather than on the part somebody had time to read. A water utility deciding which pipes to replace this year can process leak records, pipe age, soil type and pressure readings across the whole city together, instead of choosing from the complaints that came in loudest.

  3. Assisting humans. Many AI systems are not built to replace a person but to hand them a shorter list. Software that sorts a day’s applications into “clearly complete”, “clearly incomplete” and “needs a human eye” leaves the officer with only the third pile, which is where their judgement was needed anyway.

  4. Remote patient monitoring and tracking contagious diseases. A patient’s condition can be followed continuously at home rather than only at the next appointment, and public health teams can compute where cases of a contagious disease are likely to rise next. Both were described in Section 8.4.2; the gain in each case is time — an alert or a supply of medicines arriving before the problem grows rather than after.

  5. Analysis of data for research and development. Research produces far more data than researchers can read. In the search for a new medicine, an enormous number of candidate substances have to be narrowed to the few worth testing in a laboratory. Software that computes which candidates are most likely to behave in the required way shortens a search that once took years.

  6. Efficiently solving complex problems. Some problems have so many possible answers that trying them one after another is hopeless — planning the route of every bus in a city, or the seating of an examination hall so that no two students with the same paper sit together. Systems built for these compute a good arrangement quickly, even when nobody can prove it is the best possible one.

  7. Speedy disaster recovery. After a flood or a cyclone, the first question is where help is needed most, and the usual channels of information have collapsed. Satellite images taken before and after can be compared automatically to compute which areas are under water and which roads are cut, and that map can reach rescue teams within hours.

  8. Performing recurring business tasks. Every organisation has work that repeats without end — issuing receipts, checking invoices against orders, updating records, generating the same monthly report. Handing this to a system frees people for work that does not repeat.

  9. Reducing the chances of manual errors. People make mistakes when a task is dull, long and repeated, and they make more of them at the end of a shift than at the start. A system copying figures from a form into a record does not become careless at four in the afternoon.

  10. Ensuring 24-hour service availability. A program does not sleep, and it performs the same at three in the morning as at eleven in the morning. A customer who wants a balance, a booking status or a duplicate bill at midnight gets it, instead of waiting for an office to open.

Table 8.3 – a person and an AI system doing the same routine, large-volume task | Feature | A person | An AI system | |—|—|—| | Volume handled in a day | limited by working hours | limited only by computing power | | Effect of tiredness | errors rise late in a shift | none | | Consistency of the result | varies from day to day | the same result for the same input | | A situation unlike anything before | can reason about it | usually produces a poor match | | Explaining the reason for a result | can explain in words | often cannot be explained easily | | Judging what is fair or kind | can weigh it | cannot weigh it at all |

Read the table in both directions. The first three rows are the reason these systems are worth building. The last three are the reason a person stays in the loop.

Q. Why is “the same result for the same input” listed as a benefit? Surely a person is more flexible.

A. Both are true, and which one you want depends on the task. For issuing a receipt, sameness is exactly right and flexibility is a fault. For deciding whether a student deserves an extra chance, it is the other way round.

Assuming a benefit arrives simply because AI was used What goes wrong: an organisation buys an AI system expecting these gains automatically, and is surprised when the work does not improve. The fix: every benefit above rests on having enough good data about the task, and on the task being one that repeats in a recognisable pattern. Where the records are thin, wrong or held in ten places that do not agree, an AI system produces confident output built on nothing. Getting the data in order is usually the larger half of the work.

8.7 Ethical Considerations in AI

Every section so far has looked at what AI systems can do. This one looks at what they should be allowed to do, and who answers for it when something goes wrong.

The question matters more for AI than for most technology, for one reason above all: these systems are increasingly placed where a decision about a person is made. Whether a loan is approved. Whether an application reaches a human reader. Whether a patrol is sent to a particular street. Whether a scan is looked at today or next week.

Two features of such systems make those decisions different from the same decisions made by a clerk at a desk. The first is size. A clerk who judges unfairly affects the people who reach that desk; a system that judges unfairly affects everyone it touches, in the same way, every single time. The second is quietness. Nobody being refused sees the computing that refused them, and often nobody inside the organisation can see it either.

Ethics is the study of what is right and wrong to do. Applied to this subject, it asks a small set of hard questions about a system before it is switched on, and keeps asking them afterwards (see Fig. 8.7-1).

Diagram

Fig. 8.7-1 – a central box labelled “an AI system makes a decision about a person” with four arrows leading out to labelled question boxes: “Is the data it learnt from fair?”, “Does it treat people equally?”, “Who is answerable if it is wrong?”, “Can the reason be explained?”

NoteDefinition

(Ethics in artificial intelligence means examining whether an AI system treats people fairly, whether its use is honest and open, and who is answerable for its effects — questions that must be settled by the people who build and use the system, because the system itself computes an output and weighs nothing.)

That last part is worth reading twice. It would be comfortable to say that a program was unfair, as though the program had a view of its own. A program has no view. Everything it does was decided by someone — which data to build it from, which output to act on, and whether to switch it on at all.

The sections that follow take the ideas in turn: bias and prejudice, then fairness and accountability, then transparency, interpretability and explainability. They are separate ideas, but in practice they arrive together, because a system nobody can explain is also a system nobody can check for fairness.

8.7.1 Bias and Prejudice in AI Systems

An AI system is built from data, and data always comes from somewhere. It was collected by particular people, in particular places, at a particular time. Whatever was easy to collect is over-represented, and whatever was difficult or was never thought of is missing. The system then computes its patterns from what it was given, and it has no way of knowing what was left out.

Bias is the name for this. It is a consistent tilt in a system’s results, caused by the data it was built from or by the way it was built, which makes it work better for some cases than for others.

NoteDefinition

(Bias in an AI system is a consistent tilt in its results — working better or worse for particular cases or groups — that comes from the data it was built from or from choices made while building it, and not from the individual case in front of it.)

Where Bias Comes From: Missing Cases

Suppose a school installs a gate that opens by recognising registered faces. The photographs used to build it were all taken during one week of collection at the school. Table 8.4 shows when they were taken.

Table 8.4 – lighting conditions of 1,000 photographs collected to build a school gate system | Lighting condition | Number of photographs | |—|—| | Bright daylight | 700 | | Cloudy daylight | 150 | | Indoor tube light | 100 | | Dusk | 40 | | After dark, under a lamp | 10 |

how much of the data covers poor light

Given: 1,000 photographs in total, of which 40 were taken at dusk and 10 after dark

Find: the share of the data taken in poor light, as a percentage

Working:

photographs in poor light = 40 + 10 = 50

percentage = (50 ÷ 1,000) × 100 = 5

5% of the data was collected in poor light

Now think about what happens on a winter evening when a student stays late for practice. The system has computed its patterns almost entirely from bright daylight images, and it has very little to go on in the dark. It will refuse more of the students who come to the gate at that hour — not because of anything about them, but because of when the photographs happened to be taken.

Diagram

Fig. 8.7-2 – on the left, a stack of images labelled “data collected: mostly bright daylight”; an arrow through a box labelled “patterns computed”; on the right, two outcome bars labelled “works well in daylight” (tall) and “works poorly after dark” (short)

Exactly this problem has been found in real systems. A study by the National Institute of Standards and Technology in the United States examined a large number of facial recognition systems and found that many of them made noticeably more errors on faces of some groups of people than on others. The cause was the same in kind as the school gate: the sets of photographs used to build those systems did not represent everyone equally.

Where Bias Comes From: Repeating Past Decisions

The second source is harder to see, because the data is not missing at all. It is complete, and it is a complete record of how people were treated before.

Imagine a program built to shortlist students for a school quiz team, using the records of who was selected in past years. If the selections in those years were made mainly from two sections of the class, that pattern is what the program computes. It will shortlist from those sections again, and the shortlist will look well justified, because it matches the record exactly. The unfairness has been copied forward and given the appearance of a calculation. Hiring systems built from a company’s past recruitment records have run into precisely this problem.

When a system treats a person according to the group they fall into rather than on their own merits, we are no longer describing a technical tilt. We are describing prejudice.

NoteDefinition

(Prejudice is judging a person by the group they are taken to belong to rather than by their own case. An AI system shows prejudice when its output for a person is shaped by such group patterns in its data.)

Q. Can bias be removed by simply not giving the system information about people’s groups?

A. No, and this is a common trap. Other columns in the data often stand in for the removed one — a locality, a school name, a spelling of a name. The pattern comes back through those columns, and now it is even harder to see.

What Can Actually Be Done

Bias cannot be switched off, but it can be looked for, and there are three habits that catch most of it.

  • Examine the data before building anything. Who is well represented in it, who is thinly represented, and who is absent altogether?
  • Test the finished system separately on each group of cases, rather than reporting one overall figure. A system can look strong overall while failing badly for a small group, because that group barely affects the total.
  • Keep a person reviewing decisions that matter, and give anyone affected a way to question the result.

None of these is a technical trick. All three are decisions taken by the people building the system — which is the subject of the next section.

8.7.2 Fairness and Accountability

Bias is something you find in a system. Fairness is something you have to decide you want, and accountability is what happens when you do not get it. These two ideas are the practical half of AI ethics.

Fairness Has to Be Chosen

At first, fairness sounds obvious: treat people in similar situations similarly, and do not let any group be worse off for reasons that have nothing to do with the matter at hand.

NoteDefinition

(Fairness in an AI system means that people in similar situations receive similar outcomes, and that no group is consistently disadvantaged for reasons unconnected with the decision being made.)

The difficulty appears the moment you try to write that down as a rule. Suppose a school builds a program to shortlist ten students for a district competition, using records of past performance. Two rules are proposed, and both sound entirely fair:

  • Take the ten highest scores in the school, whichever section they come from.
  • Take the top two from each of the five sections.

The first rule treats every student by the same measure. The second makes sure no section is shut out because it had fewer chances to prepare. They are both defensible, they will produce different lists, and no calculation can settle which is right. That choice belongs to the people running the competition, and it must be made before the program is built — not discovered afterwards from whoever was left out.

Q. If two fair-sounding rules give different lists, how does anyone decide?

A. By deciding what the shortlist is for, and saying so openly. If the aim is to send the strongest possible team, the first rule fits. If the aim is to give every section a route in, the second fits. The mistake is not choosing a rule; it is pretending the program chose one for you.

Accountability: Someone Has to Answer

Now suppose the shortlist comes out badly — a strong student is left off, and the reason turns out to be an error in how the records were entered years ago. Who answers for that?

The tempting reply is that the program did it. That reply is not available, for the reason given in Section 8.7: a program computes an output and weighs nothing. Every step that led to the shortlist was taken by a person. Somebody chose which records to use. Somebody built the system and decided when it was ready. Somebody decided to act on its output without checking it.

NoteDefinition

(Accountability means that named people and organisations are answerable for what an AI system does, including its errors, and that this responsibility cannot be transferred to the system itself.)

Fig. 8.7-3 shows where responsibility sits along the chain.

Diagram

Fig. 8.7-3 – four boxes in a row joined by arrows, labelled “data collected”, “system built and tested”, “system put into use”, “decision affects a person”. Beneath each of the first three, a smaller labelled box: “who collected it, and what was left out”, “who built it, and how it was tested”, “who chose to use it, and who reviews its results”. An arrow curves from the last box back to all three, labelled “answerable”

Q5. If an AI system treats one group of people unfairly, whose mistake is it?

A. It belongs to people, not to the system. It may lie with whoever collected data that left that group out, with whoever built and tested the system without checking it separately for that group, or with the organisation that chose to act on its output without review. Usually it is shared among all three. “The computer decided” is never an answer, because the computer computed an output and nothing more.

What Accountability Looks Like in Practice

An organisation that takes this seriously does four things, and none of them is complicated:

  • It records how the system was built — where the data came from, what was left out, and how the system was tested — so that a question about a decision can actually be traced.
  • It names who is answerable for the system’s results, in the same way a department has a head.
  • It gives anyone affected a plain way to question a decision and have a person look at it.
  • It keeps checking after the system is in use, because the world changes and a system built on last year’s data slowly stops fitting this year’s.

Your class wants a program to choose five students for a school trip, using records of who has taken part in school activities before. Write down two selection rules that both sound fair, then work out one student who would be chosen by one rule and missed by the other. Discuss which rule your class would want, and why.

8.7.3 Transparency, Interpretability and Explainability

A municipal office receives more water-supply complaints in a week than its repair teams can attend to, so it uses a program to order them. Each complaint is given a position in the queue, computed from the details entered on the form and from records of past complaints.

A household near the end of the queue asks the office a reasonable question: why are we so far down the list? Three different things have to be in place before anyone can answer, and they are the three ideas in this section.

Transparency: Saying That the System Exists

The first is the simplest, and the most often skipped. The household should have been told, when they filed the complaint, that the order of attention is computed by a program, what information it uses, and who runs it. If nobody knows a system is there, nobody can question it.

NoteDefinition

(Transparency means being open about an AI system — that it is being used, what it is used for, what data it works from, who is responsible for it, and how it was tested.)

Transparency costs almost nothing and prevents a great deal of trouble. A notice on a form, a line on a website, a plain statement at the start of a chatbot conversation — that is most of it.

Interpretability: Seeing How the System Works

The second question is different: can a person follow the working itself? If the queue position is computed by a short set of scoring steps — so many points for the number of households affected, so many for how long the supply has been out, so many for a school or hospital on the same line — then any office clerk can read those steps and follow them.

NoteDefinition

(Interpretability is the extent to which a person can follow the internal working of a system and see how it turns an input into an output.)

Not every system is like this. A large neural network holds millions of adjustable numbers, and no person can read them and follow the reasoning, because there is no reasoning written anywhere to read. A system of this kind is often called a black box — you can see what goes in and what comes out, but not what happens in between (see Fig. 8.7-4).

Diagram

Fig. 8.7-4 – on the left, a box with “input” entering and “output” leaving, the box itself shaded solid and labelled “black box – working cannot be followed”. On the right, the same box with output leaving, plus a side panel listing three lines under the heading “main reasons for this output”

Explainability: Giving the Reason for One Result

The third idea rescues the situation. Even when the working cannot be followed step by step, it is often possible to say which parts of the input mattered most to a particular result — and for the household at the office counter, that is the answer they actually wanted.

NoteDefinition

(Explainability is the ability to give an understandable reason for a particular output of a system, in terms a person affected by it can follow.)

For the water complaint, an explainable system produces something like: this complaint is low in the queue mainly because it affects a small number of households, and because supply in the area was restored within a day on the last two occasions. That is checkable. If the household knows twenty houses are affected, not two, they now have something specific to correct.

The same idea appears elsewhere. Software that marks a region of a medical scan can show the doctor which part of the image most influenced the marking. A system that holds back an email can state which features of the message it matched.

Table 8.5 – interpretability and explainability | Feature | Interpretability | Explainability | |—|—|—| | Question it answers | how does this system work in general? | why did it give this particular result? | | Who usually needs it | the people who build and check the system | the person the result is about | | Depends on | how the system is built | tools that examine the input and the output | | Possible for a large neural network? | rarely | usually, at least in part | | Example | a scoring rule anyone can read | the two main reasons a complaint ranked low |

Assuming the most accurate system is always the right one to use What goes wrong: when choosing between two systems, an organisation picks the one that scores better on tests, without asking whether anyone can explain its results. The fix: where a decision affects a person, the ability to explain it is part of what makes the system usable at all. A slightly less accurate system whose results can be explained and challenged may serve people better than a stronger one that can only be argued with by saying “the computer said so”.

These three ideas close the chapter for a reason. Everything earlier — the recognition, the prediction, the benefits across every field — rests on systems that people must be able to see, check and question. A technology used on people, which cannot be explained to them, has a problem no amount of accuracy will fix.

8.8 Summary

  1. An algorithm is a set of steps written in a fixed order for a machine or a person to follow. Ordinary programs are built entirely from such steps.
  2. Some tasks — recognising handwriting, a face or a diseased leaf — have too many exceptions to be written as fixed steps.
  3. Artificial intelligence builds computer systems that handle such tasks by computing patterns from data instead of being given the answer for every case.
  4. Data means the facts and examples a system is given: pictures, recordings, numbers and past records.
  5. Working automatically is not the same as being AI. The question is whether the machine was given the rules or was given examples and left to compute the pattern.
  6. AI systems cannot produce genuinely new ideas, do not feel emotions, have no common sense or morals, and cannot switch to a new task without being built again for it.
  7. The name “artificial intelligence” comes from John McCarthy, who used it in the proposal for a workshop held at Dartmouth College in 1956.
  8. Early AI wrote knowledge down as rules. This worked only in narrow areas, and periods of lost funding and interest, called AI winters, followed in the 1970s and the late 1980s.
  9. Machine learning improves a program’s output by computing patterns from data. Artificial neural networks, and later deep learning with many layers, made this practical once large datasets and fast hardware arrived.
  10. The Turing Test asks whether a human judge, holding a text conversation with a machine and a person, can reliably tell which is the machine.
  11. Passing the test shows that the imitation is good. It does not show that the machine understands anything, and many useful AI systems would fail it because they were never built to converse.
  12. Across every field, AI does the same three jobs: recognition, prediction, and handling routine repeated work.
  13. In commercial work it forecasts demand, recommends products, answers routine questions and checks payments for fraud; in industry it inspects quality, predicts maintenance and supports cobots working beside people.
  14. In medical and health care it marks regions of a scan for a doctor to examine, supports remote patient monitoring, and helps track contagious diseases. The diagnosis and the responsibility remain with the doctor.
  15. In defense it flies unmanned aerial vehicles, sorts surveillance images, supports planning and defends networks. Weapons that select targets without a person involved raise serious unsettled ethical questions.
  16. In banking it computes a risk score for each payment, supports credit scoring and algorithmic trading, and answers routine customer queries.
  17. In entertainment it ranks a catalogue for each viewer, generates subtitles and controls characters in games, with the side effect of narrowing what a viewer is offered.
  18. In transport it times traffic signals from live counts, plans routes, matches rides and powers driver assistance features, which assist a driver who remains fully in charge.
  19. In security it detects phishing and malware, verifies identity by face or fingerprint, and supports surveillance — the use that needs the most care.
  20. In agriculture it finds disease early from leaf photographs, sprays only weed patches, guides irrigation and equipment, and monitors livestock.
  21. In daily life it appears in online shopping, search engines, chatbots, voice assistants, entertainment portals, facial recognition, driver assisting vehicles, and augmented and virtual reality.
  22. The main benefits are automation, smarter decisions, assisting people, remote patient monitoring and disease tracking, analysis of research data, solving complex problems, speedy disaster recovery, handling recurring tasks, fewer manual errors, and round-the-clock availability.
  23. These benefits depend on having enough good data about a task that repeats in a recognisable pattern. Without that, the system produces confident output built on nothing.
  24. Bias is a consistent tilt in a system’s results, caused by missing cases in the data or by past unfair decisions recorded as correct answers. Prejudice is judging a person by a group rather than by their own case.
  25. Bias is reduced by examining the data before building, testing the system separately on each group of cases, and keeping a person reviewing decisions that matter.
  26. Fairness must be chosen by people. Rules that all sound fair can produce different results, and the choice has to be made and stated openly before a system is built.
  27. Accountability means named people and organisations answer for what a system does. Responsibility cannot be passed to the system itself.
  28. Transparency means saying openly that a system is in use and what it works from. Interpretability is how far a person can follow the internal working. Explainability is giving an understandable reason for one particular result.
  29. A large neural network is often a black box — its input and output can be seen, but not its internal working — which is why explainability matters so much where decisions affect people.

8.9 Key Terms

  1. Accountability – named people and organisations answering for what an AI system does, including its errors.
  2. AI winter – a period in which funding for and interest in artificial intelligence fall sharply, as happened in the 1970s and the late 1980s.
  3. Algorithm – a set of steps written in a fixed order so that a machine or a person can follow it and reach a result.
  4. Algorithmic trading – the buying and selling of shares by programs that compute decisions from market data faster than a person could act.
  5. Artificial intelligence (AI) – the branch of computer science that builds computer systems able to carry out tasks normally needing human intelligence, by computing patterns from data.
  6. Artificial neural network – a program built from many simple units connected in layers, where the numbers on the connections are adjusted as examples are processed.
  7. Augmented reality (AR) – the placing of computer-generated objects into a view of the real world, usually through a phone or headset camera.
  8. Bias – a consistent tilt in a system’s results, coming from the data it was built from or from choices made while building it.
  9. Black box – a system whose input and output can be seen but whose internal working cannot be followed.
  10. Chatbot – a program that answers typed questions by matching a message to the questions it was built for.
  11. Cobot – short for collaborative robot; a robot built to work in the same space as people rather than behind a safety cage.
  12. Credit scoring – computing how likely a person applying for a loan is to repay it, from their record of earlier borrowing and repayment.
  13. Cybersecurity – the protection of computers and networks from attack, including AI systems that flag activity far outside a network’s normal pattern.
  14. Data – the facts and examples given to a program, such as pictures, recordings, numbers and past records.
  15. Deep learning – the training of neural networks with many layers stacked one after another.
  16. Demand forecasting – computing how much of an item is likely to sell in a coming period.
  17. Ethics – the study of what is right and wrong to do; in AI, the questions of fairness, openness and responsibility raised by a system’s use.
  18. Explainability – the ability to give an understandable reason for one particular output of a system.
  19. Facial recognition – computing whether a face in a camera image matches a face already registered.
  20. Fairness – the principle that people in similar situations receive similar outcomes, and that no group is consistently disadvantaged for unconnected reasons.
  21. Interpretability – the extent to which a person can follow the internal working of a system and see how it turns an input into an output.
  22. Machine learning – an approach in which a program improves its output on a task by computing patterns from data rather than being programmed with the answer.
  23. Precision irrigation – computing how much water each part of a field needs, from soil moisture readings and weather forecasts.
  24. Predictive maintenance – computing when a machine is likely to need repair, from sensor readings taken while it runs.
  25. Prejudice – judging a person by the group they are taken to belong to rather than by their own case.
  26. Recommendation system – a program that computes which items a person is likely to want, from what similar people chose and what that person looked at before.
  27. Remote patient monitoring – the continuous checking of a patient’s condition at home using connected devices.
  28. Transparency – being open about an AI system: that it is in use, what it is used for, what data it works from, and who is responsible for it.
  29. Turing Test – a test in which a human judge holds a text conversation with a machine and a person, and the machine passes if the judge cannot reliably tell which is which.
  30. Unmanned aerial vehicle (UAV) – an aircraft flown without a pilot on board.
  31. Virtual reality (VR) – the replacing of the real view entirely with a computer-generated one, seen through a headset.
  32. Voice assistant – a program that takes a spoken instruction, converts it to text, matches it to an action and carries that action out.

8.10 Objective Questions

  1. (The name “artificial intelligence” was used in the proposal for a workshop held at Dartmouth College in the year)

    1. 1943
    2. 1950
    3. 1956
    4. 1969
  2. (Fill in the blanks.)

    1. A set of steps written in a fixed order for a machine or a person to follow is called an ____________.
    2. A period in which funding for and interest in AI fall sharply is called an ____________.
    3. A system whose input and output can be seen, but whose internal working cannot be followed, is often called a ____________.
  3. (State whether each statement is true or false.)

    1. A washing machine running on a timer is an example of artificial intelligence.
    2. In the Turing Test, the judge is told in advance that one of the two respondents is a machine.
    3. A system that reports a strong overall result may still work badly for a small group of cases.
    4. Driver assistance features remove the need for the driver to watch the road.
  4. (Match each term in the left column with its meaning in the right column.)

    Term Meaning
    a. Transparency i. how far a person can follow the internal working of a system
    b. Interpretability ii. giving an understandable reason for one particular result
    c. Explainability iii. named people answering for what a system does
    d. Accountability iv. being open that a system is in use and what it works from
  5. (Name the three shapes of job that AI systems carry out in almost every field.)

  6. (Software marks a region of a chest X-ray for attention. Who gives the diagnosis?)

    1. the software, since it found the region
    2. the doctor, who examines the scan and the patient
    3. the company that built the software
    4. nobody, as a marked scan needs no diagnosis
  7. (For each description below, write whether it is an example of interpretability or of explainability.)

    1. A scoring rule short enough for any clerk in the office to read and follow.
    2. A statement that a complaint ranked low mainly because it affected few households.

8.11 Concept Questions

  1. (Define artificial intelligence. Give one example of a task that cannot be written down as a fixed set of rules, and explain briefly why it cannot.)

  2. (Differentiate between bias and prejudice in an AI system, giving one example of each.)

  3. (Describe how the Turing Test is carried out. State one thing that passing the test does not prove.)

  4. (Explain why responsibility for an unfair result cannot be passed to the AI system that produced it.)

  5. (A bakery sold 36, 44, 40, 38, 46 and 42 loaves on six working days. Calculate the average number of loaves sold in a day, showing your working. Then give one reason why this average would be a poor estimate for the day of a local festival.)

  6. (A school builds a face-recognition gate using photographs collected only in bright daylight. What would happen when a student arrives after dark? Name two steps the school could have taken to avoid this.)

8.12 Case-Based Questions

  1. (A district hospital installs software that examines each chest X-ray as it is taken and places the day’s scans in an order, so that scans showing a possible problem are read by a doctor first. The software was built from scans collected at one large city hospital.)

    1. Which of the three shapes of job described in this chapter is the software carrying out? Give a reason for your answer.
    2. Who remains responsible for the diagnosis of each patient, and why?
    3. State one check the hospital should carry out before using the software on its own patients, and explain what problem that check would catch.
  2. (A farmer has a field of area 2,500 m². A survey shows weed patches covering 400 m² in total. The weedkiller is applied at the rate of 1 litre for every 100 m² treated. The farmer is comparing an ordinary sprayer, which treats the whole field, with a camera-fitted sprayer that opens its nozzles only above a weed patch.)

    1. Calculate the quantity of weedkiller needed by each sprayer, showing your working.
    2. Calculate the saving as a percentage of the quantity the ordinary sprayer would use.
    3. State one limitation of the camera-fitted sprayer, and one benefit of the saving other than the money saved.