← Back to Subjects ororo.world

Chapter 11: Decision Making in Humans and Machines

Every morning, Meera helps at her family’s fruit stall before school. A crate of mangoes arrives, and she has to sort them: the ripe ones in front, the firm ones at the back, the bruised ones kept aside. She does it quickly. She lifts a mango, turns it once, presses it gently with her thumb, and drops it into the right basket. Ask her how she knew, and she will say she just knew. She has handled mangoes since she was six.

A few streets away, a packing unit sorts the same fruit in a very different way. Mangoes roll along a moving belt under a camera and a small weighing plate. For each mango, the system records a weight and a colour, checks those two numbers against fixed limits, and pushes the fruit into one of three chutes. No thumb, no memory, no feeling.

Two sorters. The same crate of mangoes. Two completely different ways of arriving at a decision.

Diagram

Fig. 11.0-1 – a two-panel illustration. Left panel: a girl at a fruit stall holding a mango, sorting into three baskets labelled “ripe”, “firm”, “keep aside”. Right panel: mangoes on a moving belt passing under a camera and a weighing plate, then dropping into three chutes. A vertical line separates the panels, labelled “person deciding” on the left and “machine deciding” on the right.

NoteCuriosity Corner

Q1. Why do two people looking at the same mango often put it in different baskets? Q2. What does it mean to say a decision is objective? Q3. Does a machine always make the fairer decision? Q4. How does Meera recognise a mango without measuring anything? Q5. How does a machine learn to tell a mango from a lemon?

This chapter will answer all five of these questions.

11.1 Decisions, Big and Small

You make hundreds of decisions in a day without noticing. Which shirt to wear. Whether to take the shorter road or the safer one. Whether to answer a question in class or stay quiet. Most of these take less than a second, and you never stop to work out how you did it.

Machines around you are settling questions too. A lift that is called from two floors at once has to pick which floor to go to first. A traffic signal has to pick which side gets the green light. A billing counter has to pick whether a discount applies to a bill. None of these happen by chance. Something goes in, something comes out, and there is a reason connecting the two.

That connection is what this chapter is about.

NoteDefinition

(Decision making is the process of choosing one option out of two or more, using some information and some reason for preferring that option.)

What Every Decision Needs

Look at any decision closely and you will find three parts.

First, there is the input — the information you start with. Input simply means whatever goes into a process. For Meera at the fruit stall, the input is what she sees and feels when she picks up a mango.

Second, there is the basis for choosing. This is the standard being applied — the criteria. A criterion is a test that an option has to pass. “Is it soft enough to sell today?” is a criterion. “Does it weigh more than 200 grams?” is a different criterion.

Third, there is the output — the choice that comes out at the end. Output means whatever comes out of a process. For Meera, the output is the basket she drops the mango into. See Fig. 11.1-1.

Diagram

Fig. 11.1-1 – a left-to-right block diagram with three boxes joined by arrows. Box 1 labelled “Input – the information available”. Box 2 labelled “Criteria – the standard used to judge”. Box 3 labelled “Output – the option chosen”. Below the diagram, two example strips: (a) “colour and softness of a mango” → “is it ready to sell today?” → “front basket”; (b) “weight reading from a plate” → “is it above 200 g?” → “chute 1”.

Q. If two people are given exactly the same information and still choose differently, which of the three parts must be different?

A. The criteria. They are starting from the same input, so the difference has to be in what standard each one is applying.

Same Question, Different Basis

Here is the part worth pausing on. Two sorters can face the same crate of mangoes and both do a sensible job, yet arrive at their answers in completely different ways.

Meera’s basis is built out of years of handling fruit. She cannot write it down as a rule, and she could not hand it to somebody else in a sentence. Ask her why one mango went to the front and another did not, and the best she can offer is that the first one felt right.

The packing unit’s basis is written down exactly. Above 200 grams and past a certain shade of yellow goes to chute 1; everything else goes to chute 2. There is no room for interpretation. Feed the same mango through twice and it lands in the same chute both times.

Neither basis is the better one in every situation. But they behave very differently, and the rest of this chapter works out how. The two ways of deciding have proper names, and the next two sections give them.

11.2 Subjective Decision Making

Think about how your class picks the design for the school notice board. Four rough sketches go up on the board, hands go up, and one of them wins. Nobody measures anything. Nobody can prove that the winning sketch was the correct answer, because there is no correct answer to prove. Each person voted for the one that appealed to them.

That is the first of the two ways of deciding, and it has a name.

Subjective decision making is the process of making decisions based on personal opinions, feelings, beliefs, and experiences. The word subjective points to the person doing the deciding — the choice comes out of who they are, not out of a measurement anyone could repeat.

NoteDefinition

(Subjective decision making means arriving at a choice using personal opinions, feelings, beliefs, and experiences rather than measurable evidence.)

11.2.1 What Makes a Decision Subjective

A decision is subjective when there is no single standard that everyone agrees on. Meera’s thumb-press has no number behind it. Your classmate’s vote for the third sketch has no scale behind it. Both are real decisions, and both are subjective.

Three things follow from having no agreed standard.

The reasoning is hard to measure

If you ask Meera to explain why one mango went to the front basket, she can talk about how it felt, but she cannot hand you a figure. The reasoning behind a subjective decision is difficult to quantify — that is, difficult to turn into a number. This is not carelessness on her part. There simply is no number there to give.

The choice depends on the situation

A subjective decision leans heavily on context: the surroundings, the social setting, and the person’s state of mind at that moment. The same student may vote for a bright, busy design on a cheerful morning and a plain one on a tired afternoon.

Different people reach different answers

Because the standard sits inside each person, what seems right to one person may seem wrong to another. Two customers can look at the same shirt and disagree completely about whether it is worth buying, and neither of them is making a mistake.

treating “subjective” as another word for “wrong” What goes wrong: Students often read subjective as careless, lazy, or unreliable, and objective as the serious, correct one. The fix: Subjective means the basis of the choice is personal, not that the choice is poor. A judge weighing whether a punishment is fair, or a doctor weighing how much pain a patient can bear, is deciding subjectively and deciding carefully at the same time.

Q. Your friend says a film was boring and you say it was excellent. Who is right?

A. Neither, because there is nothing here to be right about. There is no agreed measurement for how interesting a film is, so two honest people can hold opposite views.

11.2.2 Where Subjective Decisions Work Well

Subjective decision making is not a fallback used when better methods are unavailable. There are whole areas of life where it is the only sensible approach.

Judgement of an artist. An artist deciding which of her paintings is the best one is working entirely on personal taste and interpretation. There is no instrument that measures beauty, so there is nothing else for her to work with.

Consumer choices. A shopper choosing between two brands of shoes is weighing up how they look, how they feel, and what she has liked before. Two shoppers with the same budget in the same shop will walk out with different boxes.

Everyday personal choices. Deciding which subject you enjoy most, which friend to sit with, or how to spend a free afternoon — none of these has a measurable right answer either.

What subjective decisions are good at

The strength of this approach is flexibility. A person can adapt to a situation nobody planned for, weigh in values that no rule mentions, and handle a case where the information is thin or missing altogether. A shopkeeper who lets a regular customer pay tomorrow is making a decision no written rule would produce.

One honest limitation

The same flexibility that makes subjective decisions adaptable also makes them inconsistent. The same person, given the same case twice, may well answer differently on the two occasions — and they will not always be able to explain the gap. In any setting where people expect similar cases to be treated similarly, that inconsistency is a real problem.

That problem is exactly what the next section is a response to.

11.3 Objective Decision Making

Suppose your school has to buy 60 notebooks and two shops have quoted prices. This time nobody’s taste comes into it. You work out what each shop charges for one notebook, compare the two figures, and order from the cheaper one. Anyone in the class doing the same sum would reach the same shop.

That is the second way of deciding.

Objective decision making is the process of making decisions based on unbiased, quantifiable, and verifiable data, facts, and evidence rather than on personal opinions, feelings, or beliefs. Data here means facts and figures that have been collected — prices, weights, timings, counts. Quantifiable means it can be expressed as a number. Verifiable means somebody else can check it and confirm it.

NoteDefinition

(Objective decision making means arriving at a choice using measurable and verifiable data, facts, and evidence, judged against consistent criteria, rather than personal opinion.)

Q2. What does it mean to say a decision is objective?

A. It means the choice rests on measurable facts that anyone can check, and on a standard that stays the same from one case to the next — not on how the person deciding happens to feel about it. Because the basis sits outside the person, a second person working from the same facts should reach the same choice.

11.3.1 What Makes a Decision Objective

Four features run through every objective decision.

The information is measurable and checkable

The facts used do not depend on who is looking. A mango’s weight is 214 grams whether Meera weighs it or the packing unit does. If two people disagree about it, they can settle the matter by weighing it again.

The criteria stay the same

Objective decision making follows a fixed set of criteria, so similar situations are handled in the same way. If the rule is “above 200 grams goes to chute 1”, then every mango above 200 grams goes to chute 1 — the first one of the morning and the ten-thousandth one of the evening alike.

The reasoning can be explained

Because the criteria are written down, the process is transparent. You can point at the rule, point at the number, and show exactly why the answer came out the way it did. Nobody has to take your word for it.

The result is repeatable

Run the same case through twice and you get the same answer twice. That is the clearest sign that the decision did not depend on a mood.

choosing a supplier for 60 notebooks

Given: Shop A sells 12 notebooks for ₹360. Shop B sells 15 notebooks for ₹435. The notebooks are of the same size and page count.

Find: which shop is cheaper per notebook.

Working:

Cost per notebook at Shop A = ₹360 ÷ 12 = ₹30

Cost per notebook at Shop B = ₹435 ÷ 15 = ₹29

₹29 is less than ₹30.

Shop B is cheaper, at ₹29 per notebook against ₹30 at Shop A.

Notice what that example did not rely on. Not which shopkeeper is friendlier. Not which shop is nearer to the bus stop. Only two numbers and one division each. Somebody who disliked Shop B would still get ₹29.

Q. The two shops quoted for notebooks of different page counts. Does the sum above still work?

A. No. The comparison only holds if the notebooks are the same in every way that matters. With different page counts you would have to compare cost per page instead, or the numbers would be measuring different things.

11.3.2 Where Objective Decisions Work Well

Objective decision making is the right choice wherever fairness, accuracy, and consistency matter more than flexibility.

Financial investments. Choosing where to invest money on the basis of financial analysis, past performance, and an assessment of risk — rather than on a hunch about which company sounds promising.

Legal judgments. Reaching a legal decision from the facts of the case, the law, and earlier decisions in similar cases, rather than from personal belief or outside pressure.

Scientific and technical work. Deciding whether a new design is better by testing both and comparing the readings.

What objective decisions are good at

The main strengths are consistency, transparency, and impartiality. Similar cases get similar treatment. The reasoning can be shown to anyone who questions it. And because personal feeling is kept out, the decision is not swayed by fear, optimism, or prejudice.

One honest limitation

Objective decision making is only as good as the data it is given. If the figures are incomplete, out of date, or simply wrong, the method will process them faithfully and produce a wrong answer with complete confidence. It also has nothing to say about factors that were never measured. The notebook sum above cannot notice that Shop B is three kilometres away and delivers late — that fact was not in the data, so it played no part in the decision.

This is worth remembering, because it is easy to assume that anything worked out from numbers must be the better answer. Sections 11.4 and 11.5 look at why humans lean towards the first method and machines towards the second.

11.4 Why Decision Making by Humans is Subjective

You now know what a subjective decision is. The question worth asking next is why human decisions keep turning out that way, even when the person is trying hard to be fair.

The answer is that a human being does not receive information the way a weighing plate does. By the time a fact reaches you, it has passed through your mood, your memory, and everything you have been taught. Several separate influences are at work, and they are listed below. See Fig. 11.4-1.

Diagram

Fig. 11.4-1 – a central circle labelled “human decision” with seven arrows pointing into it from around the edge, each arrow labelled with one influence: emotions, personal experience and background, cognitive biases, values and beliefs, intuition, context and environment, other people’s opinions.

Influence of emotions

Human decisions are shaped by emotions such as fear, joy, anger, and sympathy. Emotion tends to push the immediate feeling ahead of the long-term outcome. A student who is anxious before a test may pick the easier question first, even though the harder one carried more of what she had prepared.

Personal experience and background

Every person’s life history shapes how they read a situation. Upbringing, the place they grew up in, and what has happened to them before all feed into the choice. Two people offered the same second-hand cycle will judge it differently if one of them has already been let down by a second-hand purchase.

Cognitive biases

Humans are prone to cognitive bias — a regular pattern of straying from careful reasoning, usually without noticing it.

NoteDefinition

(A cognitive bias is a systematic error in judgement that a person makes without being aware of it.)

Two well-known ones are worth naming. The availability bias is the habit of leaning on whichever example comes to mind first, rather than on how common that example actually is. The anchoring bias is the habit of leaning too heavily on the first piece of information received — the first price quoted, say, which then colours every price heard afterwards.

Values and beliefs

Decisions are guided by what a person believes is right. These beliefs vary widely from person to person, which is why two people can be handed identical facts and still walk away with opposite conclusions about what should be done.

Intuition and gut feelings

Often a person acts on intuition — a sense of what to do that does not come from any reasoning they can spell out. Meera’s thumb-press is intuition at work. Intuition can be very good, particularly in someone with long practice, but it is subjective by its nature, because it is built out of that one person’s experience.

Context and environment

The setting matters. Social surroundings, cultural expectations, and even the room a person is sitting in can change the answer. Someone may go along with what the group is choosing, and choose something else entirely when deciding alone.

Interpersonal influence

Finally, decisions are shaped by the opinions and advice of family, friends, and colleagues. A student picking a subject to study rarely picks it in silence.

Q1. Why do two people looking at the same mango often put it in different baskets?

A. Because neither of them is reading a number off the mango. Each is applying a standard built out of their own experience, habits, and mood at that moment, so the same fruit can genuinely look ready to one person and not quite ready to the other.

Q. If a person is careful and honest, can they remove all of these influences from a decision?

A. No. Emotions and cognitive biases work below the level of awareness, so trying harder does not switch them off. What careful people do instead is write down their criteria in advance and check their answer against evidence — which is a way of borrowing from the objective method.

Subjectivity is not a fault

It would be easy to read that list as seven weaknesses. It is not. The same influences that make human decisions inconsistent are what let a person handle a situation nobody prepared them for, weigh in a value that no rule mentions, and notice that something about a case is unusual.

A rule cannot do that. Which brings us to what happens when the deciding is handed to a machine.

11.5 Why Decision Making by Machines is Objective

A machine has no history, no mood, and nobody it wants to please. Whatever it produces has to come from what was put into it and the instructions it was given. That single fact is the reason decision making by machines is objective in nature.

Below are the main reasons, and then one warning that matters more than any of them.

Data-driven approach

Machines arrive at decisions from data inputs — readings, records, counts, prices. A system can work through a very large amount of data and pick out patterns and trends in it that a person going through the same records by hand would take weeks to spot.

Predefined rules and algorithms

Every machine decision runs on instructions written in advance. The general name for such a set of instructions is an algorithm.

NoteDefinition

(An algorithm is a finite set of clearly stated steps that takes an input and produces an output.)

Because the steps are fixed and unambiguous, the same input always produces the same output. Here is the packing unit’s rule set out properly.

sorting a mango on the belt Input: the weight of one mango in grams, and its colour reading from the camera Output: the chute the mango is pushed into Steps:

1. Read the weight of the mango.
2. Read the colour of the mango.
3. If the weight is 200 g or more AND the colour is in the ripe range,
   send the mango to chute 1.
4. Otherwise, if the weight is 200 g or more,
   send the mango to chute 2.
5. Otherwise, send the mango to chute 3.

Why it works: every mango falls into exactly one of the three cases, so the rule always has an answer and never has two.

See Fig. 11.5-1 for the same rule drawn as a flowchart.

Diagram

Fig. 11.5-1 – a flowchart. Start box “mango arrives on belt” → process box “read weight and colour” → decision diamond “weight ≥ 200 g?” → “No” branch goes to box “chute 3”; “Yes” branch goes to a second decision diamond “colour in ripe range?” → “Yes” branch to box “chute 1”, “No” branch to box “chute 2”. All three chute boxes join into an end box “next mango”.

No emotions and no personal bias

Machines do not experience fear, optimism, affection, or prejudice, so none of these can push the answer one way or the other. In areas where impartiality is important, such as legal or financial work, that emotional neutrality is a real advantage.

Consistency and repeatability

Given the same inputs and the same conditions, a machine produces the same decision every single time. It does not get tired at four in the afternoon and it does not treat the last mango of the day differently from the first. This repeatability is the clearest mark of an objective process.

Quantitative analysis

Machines work with quantities. Statistical methods and mathematical models are objective by their nature: they take numbers in and give numbers out. A system built to forecast share prices, for example, works from recorded prices and trading volumes. It has no way of being excited by a rumour, because a rumour is not a number it was given.

Q. The packing unit’s rule sends a slightly bruised 250 g mango to chute 1 along with the perfect ones. Whose mistake is that?

A. Not the machine’s. It applied its rule correctly. The rule never asked about bruising, so bruising played no part. A machine’s decision can only be as complete as the criteria someone wrote into it.

Impartiality and fairness — with a warning

Because a machine works from stated rules and recorded data, it can be built to treat every case by the same standard. This is why such systems are used in places like shortlisting job applications, where the hope is that a fixed set of criteria will keep personal prejudice out of the process.

That hope holds only when the rules and the data are sound. It is not automatic.

A machine has no prejudice of its own, but it faithfully repeats whatever prejudice is present in the rules it was given or the records it was trained on. If past hiring records mostly contain one kind of candidate, a system built from them can carry that imbalance forward and reject good applicants — and it will do so consistently, which makes it harder to notice than a single unfair interviewer. Anyone deploying such a system has to check who it accepts and rejects, and keep a person answerable for the outcome.

Q3. Does a machine always make the fairer decision?

A. No. A machine is consistent, which is not the same thing as fair. It applies the criteria it was given to the data it was given, so if either of those is flawed, the machine produces an unfair result every time without hesitating. Fairness has to be built in by the people who choose the criteria and the data, and then checked afterwards.

One more point, and it applies to everything in this section. A machine does not weigh up a mango the way you would. It compares numbers against stored limits and follows the branch that matches. The word decision is convenient, but what is actually happening is computation.

11.6 Humans and Machines Side by Side

Two methods have now been described separately. Putting them next to each other makes the differences sharper, and also makes it clearer that neither one wins outright.

11.6.1 A Direct Comparison

Table 11.1 sets the two side by side on the points that matter most in practice.

Table 11.1 – subjective decision making by humans vs objective decision making by machines

Feature Subjective decision making (humans) Objective decision making (machines)
Basis of the choice Personal opinions, feelings, beliefs, experience Measurable and verifiable data, facts, evidence
Where the criteria come from Inside the person, often unstated Written down in advance as rules or an algorithm
Same case decided twice May come out differently Comes out the same every time
Can the reasoning be shown? Often hard to put into words Can be traced step by step
Handling a situation nobody planned for Handles it by judgement Has no rule for it, so it fails or forces a wrong match
Volume of information handled Limited by attention and memory Very large volumes, quickly
Influenced by mood or tiredness Yes No
Main risk Bias and inconsistency Bad data or badly chosen criteria, applied faithfully

Read the last row carefully, because it is the row students most often skip. Both methods can produce a wrong answer. They just fail in different ways. A person fails unpredictably, one case at a time. A machine fails predictably, in exactly the same way, on every case it touches — which can make the damage larger before anyone spots it.

Q. Which of the two methods is more accurate?

A. Neither, as a general rule. Accuracy depends on the question. For “which supplier is cheaper”, the machine’s method is far more reliable. For “which of these two designs suits our school better”, there is no measurement to be accurate about.

11.6.2 Choosing the Right Approach

The useful question is not which method is better, but which one fits the decision in front of you.

When the subjective approach is preferred

Choose it when the matter is personal, creative, ethical, or emotional; when the available data is thin, missing, or beside the point; or when the decision has to reflect somebody’s values and preferences. Selecting the theme for the school annual day, deciding how to comfort a friend, or judging an art competition all belong here. Trying to force a number onto these questions does not make the answer better — it just hides the judgement inside a formula.

When the objective approach is preferred

Choose it when the decision has to rest on concrete evidence, and when impartiality and accuracy are what matter. Scientific research, financial planning, and the day-to-day running of operations are the usual examples: which fertiliser gave a better yield, whether a loan repayment fits within an income, how many buses a route needs. Here consistency is not a limitation, it is the whole point.

Most real decisions use both

In practice the two methods are rarely used alone. A medical team choosing a treatment plan reads test results, dosage limits, and recorded outcomes — all objective — and then weighs how much the patient can tolerate and what the patient wants, which is not measurable at all. A court works from the facts of the case and the law, and still relies on judgement about what a fair sentence is.

A school picking four students for a science quiz team works the same way. Marks in a written round narrow the field objectively. Choosing among the students who all scored well — who works steadily under pressure, who explains ideas clearly to the others — is judgement, and no test produces that number.

The sensible pattern is to let the objective method do what it does well, and keep a person answerable for the part that needs judgement. That way you get consistency where consistency helps, and flexibility where a rule would have been blind.

So far, everything has been about deciding between options that were already named — this basket or that one, this shop or that one. But before Meera can decide which basket a mango goes in, she has to work out that the thing in her hand is a mango. That step turns out to be much harder than it looks, and it is where the rest of the chapter goes.

11.7 How Do Humans Identify Objects?

Before any decision about a mango can be made, one question has to be settled: is this a mango? You answer questions like that all day without noticing you are answering them. You spot your water bottle on a crowded table. You pick your own bag out of thirty in the corridor. You know at a glance that the round green thing on the shelf is a lemon and not a lime.

Sorting a thing into the right category is called object classification.

NoteDefinition

(Object classification means deciding which known category a given object belongs to — for example, deciding that a particular fruit is a mango rather than a papaya.)

For a person this happens in a fraction of a second, which makes it easy to assume it is simple. It is not. Four things are going on at once.

Sensory input

Humans identify objects using several senses together — sight, touch, smell, and sometimes taste. You might recognise a banana by its yellow colour, its curved shape, and its familiar smell, without consciously listing any of those. See Fig. 11.7-1.

Diagram

Fig. 11.7-1 – a ring of five labelled icons (eye, hand, nose, ear, tongue) with arrows pointing inward to a central outline of a head holding a mango. Caption text inside the centre: “several senses, one answer”.

Using more than one sense at a time is a real advantage. If the light is poor and you cannot see the fruit clearly, touch and smell still carry enough information to settle the question.

Contextual recognition

People recognise things in conditions they have never met before. You know a half-eaten apple is an apple. You know a mango is a mango when it is cut into slices, when it is lying in a different room, when half of it is hidden behind a cloth, or when it is upside down. None of those views matches the picture of a whole mango you would draw from memory, and it makes no difference.

Learning and memory

Recognition is built out of past experience. Once a child has learnt what an orange looks like, that knowledge carries across to oranges of different sizes and slightly different shades, without anyone having to teach each one separately. This is why Meera is fast: her memory holds thousands of mangoes.

Subjectivity

Human identification is affected by personal preference, familiarity, and even mood. Someone who eats a fruit often will recognise it faster and more reliably than someone seeing it for the third time in their life. Two people can also disagree about a borderline case — is that a small mango or a large one? — for the same reasons that made their sorting decisions differ back in Section 11.4.

Q4. How does Meera recognise a mango without measuring anything?

A. She is matching what she sees, feels, and smells against a memory built from years of handling mangoes. Nothing is measured because nothing needs to be: the shape, the colour, the weight in her palm, and the smell together match a pattern she already holds, and several senses agree at once. That memory also lets her recognise a mango she has never seen before — a different variety, a different size, or one lying half covered.

Q. Why can you recognise your own bag in a corridor full of similar bags?

A. Because you are not matching the general idea of a bag. You are matching small particular details you have seen many times — a worn strap, a mark near the zip — which is memory doing the work rather than the eye alone.

One honest limitation

Human recognition is flexible, but it is neither consistent nor quick at scale. Ask a person to identify ten thousand fruits one after another and the errors will start creeping in from tiredness alone. People can also be fooled by a good imitation: a wax fruit on a display table catches most of us at first glance, because it satisfies everything the eye was checking.

A machine has the opposite profile, as the next section shows.

11.8 How Do Machines Identify Objects?

A machine has no childhood spent around fruit. It cannot smell anything and it cannot pick a mango up. All it receives is an image: a grid of coloured dots. Somehow it has to turn that grid into the word mango.

Nobody solves this by writing a rule such as “if it is yellow and oval, it is a mango”. That rule fails on a green mango, fails on a lemon, and fails on a mango photographed from the end. Fruit does not come in shapes tidy enough to be written down. So the problem is approached from the other direction: instead of writing the rule, the system is given a large number of examples and the pattern is worked out from them.

11.8.1 Learning from Labelled Data

The starting point is a dataset — an organised collection of data kept together for a purpose. For object classification, the dataset is made of images, and each image carries a label saying what it shows. Table 11.2 shows what a very small piece of such a dataset looks like.

Table 11.2 – a small labelled image dataset for fruit classification

Image file Label
fruit_001.jpg mango
fruit_002.jpg lemon
fruit_003.jpg mango
fruit_004.jpg papaya
fruit_005.jpg lemon
fruit_006.jpg mango
fruit_007.jpg papaya
fruit_008.jpg lemon

A real dataset for this job holds thousands of such rows, not eight, and covers ripe fruit, unripe fruit, different varieties, and different lighting. The images used to build the system are called the training data.

The system does not store these pictures and compare new fruit against them one by one. What it does is measure things about each image — the edges in it, the outlines, how the colours are spread out, how rough or smooth the surface looks. Each such measurable property is called a feature. Pulling these properties out of an image is known as feature extraction. One widely used kind of model for this is a convolutional neural network (CNN), which builds up from small edges to larger shapes across several stages.

Once features have been extracted from thousands of labelled images, the system settles on which combinations of features go with which label. A new photograph is then handled the same way: features are extracted from it, compared against those learnt combinations, and the closest matching label is produced. See Fig. 11.8-1.

Diagram

Fig. 11.8-1 – a two-row pipeline. Top row, labelled “training”: a stack of images marked “mango / lemon / papaya” → box “extract features (edges, outlines, colour spread, texture)” → box “learn which features go with which label”. Bottom row, labelled “using it”: a single new image with a question mark → box “extract features” (arrow from the learnt box above pointing down into it) → output box “label: mango”.

image classification by a machine What it is: a system that takes an image of an object and produces the name of the category it belongs to. How it works: features such as edges, outlines, colour distribution, and texture are extracted from thousands of labelled training images; the system settles on which feature combinations match which label; a new image is then given the label whose feature combination it matches most closely. Used for: sorting fruit on a packing line into grades without a person handling each piece. Watch out: it can only produce labels it was trained on, and it has no way of telling you when a new image is unlike anything it has seen.

Q5. How does a machine learn to tell a mango from a lemon?

A. It is shown a large number of images that have already been labelled by people as mangoes or lemons. From these it extracts measurable features — outline, colour spread, surface texture — and works out which combinations of features go with which label. When a new image arrives, it extracts the same features and gives the image the closest matching label. Nothing about mangoes is explained to it in words; the pattern comes entirely from the labelled examples.

thinking the machine keeps a picture of every mango and looks it up What goes wrong: Students often picture the system flipping through a stored album until it finds a matching photograph. The fix: Storing every possible mango is impossible, and it would fail on the very first mango not in the album. What is stored is a set of learnt feature combinations, which is why a mango the system has never seen can still be labelled correctly.

Q. Where does the word “mango” in the answer actually come from?

A. From the people who labelled the training images. The system matches patterns and returns whichever label was attached to that pattern. It has no meaning for the word beyond that.

11.8.2 Where Machines Struggle

Machines are excellent at the part humans find tiring. They are consistent — the same image gives the same label every time, with no drift from mood or fatigue — and they can work through images far faster than any person. Where they fall short is exactly where humans were strong.

Unfamiliar situations

A system may struggle with an object presented in a way it has not met before: an apple cut into pieces, a banana half peeled, a mango lying under a cloth. It relies heavily on its training data, and it does not generalise well beyond it.

Changes in the picture itself

Variations in lighting, camera angle, and objects partly blocking one another all shift the features that get extracted. A photograph taken in dim evening light can produce a different colour spread from the same fruit in the morning, and the label can change with it.

Dependence on the training data

The accuracy of the system depends on the quality and variety of the data it was trained on. If the dataset is small, one-sided, or missing a common variety, the system will misclassify that variety every time it appears. It will do so confidently, because it has no way of representing “I have not seen this before”.

The same technique is used to identify people, not only fruit, and the failure pattern is the same: a system trained mostly on one kind of face performs worse on faces unlike those, and the people it fails on are the ones least able to argue with it. Where such a system is used at all, its results should be treated as a suggestion for a person to check, never as the final word about someone.

Write down the shortest list of features you think would let a machine tell a lemon from a mango. Then find one real fruit, or one photograph, that your list would label wrongly, and add the feature that would fix it.

11.9 Object Classification in Humans and Machines Compared

Both Meera and the packing line answer the same question — what is this object? — and both answer it well. They just get there by opposite routes, and they break down in opposite situations. Table 11.3 puts the two together.

Table 11.3 – object classification by humans vs by machines

Feature Humans Machines
How the ability is gained Everyday experience over years Training on a large set of labelled images
Information used Several senses together — sight, touch, smell, sometimes taste Features extracted from an image, such as edges, outline, colour spread and texture
Object partly hidden or cut up Usually still recognised Often misclassified, because it does not match the training images
Same object shown twice May be judged differently Given the same label both times
Speed and volume Slows down and tires Thousands of images, at a steady rate
Change in light or angle Handled easily Can change the extracted features, and so the label
Main source of error Tiredness, preference, familiarity Gaps or imbalance in the training data
Reaction to something never seen before Notices it is unfamiliar and says so Returns its closest label, with nothing marking it as a guess

The last row is the one worth carrying out of this chapter. A person who is handed an unfamiliar fruit will say they do not recognise it. A classification system has no such option — it was built to produce a label, so it produces the nearest one it holds. Nothing in the output shows that the case was outside anything it had learnt.

Q. A packing line labels a guava as a mango. Would the same system make that mistake again tomorrow?

A. Yes, on the same image under the same conditions. Its behaviour does not drift, so the error repeats until someone changes the training data or the criteria. A person making that mistake might well get it right the second time — which is the difference between inconsistent and consistent failure.

Why both are still needed

Real packing units use both, and so do most systems that classify anything important. The machine handles the volume, because no team of people can inspect ten thousand mangoes an hour at a steady standard. A person stays in the loop for the cases the machine was never shown: the odd variety, the damaged crate, the day the light in the shed changes.

That arrangement matches everything in this chapter. Use the objective method where consistency and volume are what the job needs. Keep human judgement where the situation is unusual, where values are involved, or where somebody has to be answerable for the result.

Meera and the packing line, in the end, are not rivals. They are good at different halves of the same job.

11.10 Summary

  1. Decision making is the process of choosing one option out of two or more, using some information and some reason for preferring that option. Every decision has an input, a set of criteria, and an output.

  2. Subjective decision making arrives at a choice using personal opinions, feelings, beliefs, and experiences. There is no single agreed standard, the reasoning is hard to measure, and different people can reach different answers from the same facts.

  3. Subjective decisions suit personal, creative, and ethical matters. Their strength is flexibility; their weakness is inconsistency.

  4. Objective decision making arrives at a choice using measurable and verifiable data, facts, and evidence, judged against consistent criteria. The reasoning can be explained and the result can be repeated.

  5. Objective decisions suit scientific, financial, and operational work. Their strength is consistency and transparency; their weakness is that they are only as good as the data and criteria supplied.

  6. Decision making by humans is subjective in nature. The main factors are emotions, personal experience and background, cognitive biases, values and beliefs, intuition, context and environment, and interpersonal influence.

  7. A cognitive bias is a systematic error in judgement that a person makes without being aware of it.

  8. Decision making by machines is objective in nature. The main reasons are a data-driven approach, predefined rules and algorithms, absence of emotions and personal bias, consistency and repeatability, quantitative analysis, and impartiality.

  9. An algorithm is a finite set of clearly stated steps that takes an input and produces an output. The same input always gives the same output.

  10. A machine is consistent, which is not the same as fair. It repeats whatever bias is present in its rules or its data, and it does so on every case.

  11. Most real decisions combine both approaches: the objective method handles what can be measured, and a person stays answerable for the judgement.

  12. Object classification means deciding which known category an object belongs to.

  13. Humans classify objects using several senses together, contextual recognition, learning and memory, and are affected by subjectivity. They handle unfamiliar and partly hidden objects easily but tire and lose consistency.

  14. Machines classify objects by extracting features from images — edges, outlines, colour spread, texture — and matching them against combinations learnt from labelled training data. They are fast and consistent but depend heavily on the quality and variety of that training data.

  15. A machine given something outside its training returns its nearest label anyway, with nothing to show that the case was unfamiliar.

11.11 Key Terms

  1. Algorithm — a finite set of clearly stated steps that takes an input and produces an output.

  2. Cognitive bias — a systematic error in judgement that a person makes without being aware of it.

  3. Criteria — the standards or tests an option has to meet before it is chosen.

  4. Data — facts and figures that have been collected, such as prices, weights, timings, or counts.

  5. Dataset — an organised collection of data kept together for a purpose.

  6. Feature — a measurable property of an object or an image, such as its outline, colour spread, or texture.

  7. Input — the information that goes into a process.

  8. Label — the name attached to an item in a dataset, saying which category it belongs to.

  9. Object classification — deciding which known category a given object belongs to.

  10. Objective decision making — arriving at a choice using measurable and verifiable data, facts, and evidence, judged against consistent criteria, rather than personal opinion.

  11. Output — whatever comes out of a process.

  12. Subjective decision making — arriving at a choice using personal opinions, feelings, beliefs, and experiences rather than measurable evidence.

  13. Training data — the labelled examples used to build a classification system before it is given anything new.

11.12 Objective Questions

  1. Fill in the blank: Choosing an option on the basis of personal opinions, feelings, beliefs, and experiences is called ________ decision making.

  2. Objective decision making is best described as decision making that relies on:

    1. the mood of the person deciding
    2. measurable and verifiable data
    3. intuition built from long experience
    4. the advice of family and friends

  3. State whether true or false: Given the same input and the same conditions, a machine may produce a different output on a second run.

  4. Which of the following is the clearest example of objective decision making by a machine?

    1. A music app suggesting songs to suit a listener’s mood
    2. A bus depot system allotting extra buses to a route using recorded passenger counts
    3. A shopkeeper offering a discount to a regular customer
    4. A panel choosing the most attractive poster for the school gate

  5. Match each term in Column A with its description in Column B.

Column A Column B
a. Algorithm i. The labelled examples used to build a classification system
b. Feature ii. A finite set of clearly stated steps that turns an input into an output
c. Label iii. The name attached to an item in a dataset
d. Training data iv. A measurable property of an image, such as its outline or texture

  1. Name the standard or test that an option must meet before it is chosen in a decision.

  2. In a camera-based fruit-sorting system, the training data is used to:

    1. store one photograph of every fruit the system will ever meet
    2. let the system settle on which feature combinations go with which label
    3. supply the system with live readings while it is running
    4. record how many fruits passed through the line each day

11.13 Concept Questions

  1. Define objective decision making, and state one situation in which it is preferred.

  2. Two teachers read the same answer script and award different opinions of its quality. Give two reasons why this can happen even when both are being careful.

  3. Differentiate between subjective and objective decision making on any three points.

  4. A school needs 90 exam pads of the same size. Shop P sells 10 pads for ₹250 and Shop Q sells 12 pads for ₹288. Work out the cost per pad at each shop, showing your steps, and state which shop the school should order from. Then state one factor this calculation cannot take into account.

  5. Explain, in your own words, how a machine identifies an object in an image.

  6. A machine applies the same criteria to every case it handles. Explain why this makes its decisions consistent but does not make them fair.

  7. What would happen if a mango-sorting system were trained only on images of ripe yellow mangoes and a crate of green mangoes then arrived on the belt? How would a person sorting the same crate behave differently?

11.14 Case-Based Questions

  1. A school has to send three students to a district science exhibition. All students who wish to go first take a written round marked out of 50. The scores are given in Table 11.4. The two highest scorers are selected straight away, and a panel of teachers then chooses the third student from the rest.

Table 11.4 – written round scores for the science exhibition

Student Score out of 50
Anil 38
Bhavna 45
Chetan 45
Divya 29
Farhan 41
  1. Name the two students selected by the written round, and state which type of decision making that round represents.
  2. The panel then picks the third student after watching how the remaining students explain their models to visitors. Which type of decision making is this, and why can it not be replaced by a second written test?
  3. State one risk the school would run if all three places were decided by the written scores alone.
  1. A mango packing unit installs a camera-based sorting system. It is trained on several thousand labelled images of one local variety of mango, photographed in the shed under its daytime lighting. Halfway through the season, a second variety arrives — smaller, and green even when ripe. The unit also begins running a night shift under tube lights.
    1. Predict what the system is likely to do with the second variety, and explain why.
    2. State one reason the night shift could affect the labels the system produces, even for the variety it was trained on.
    3. Suggest two steps the owner should take so that the unit keeps sorting correctly, and say which of the two protects the fruit that arrives tomorrow.