A4.4 · Book pp. 274–278
Ethical considerations
A model can be accurate and still do harm. This section asks who benefits, who is hurt, and who answers for it.
What you need to know
- A4.4.1 Discuss the ethical implications of machine learning in real-world scenarios.
- A4.4.2 Discuss ethics as technologies become integrated into daily life.
⚠️ Common mistakes
- Do not reduce complex issues to right or wrong. “Discuss” means weighing up both sides.
- Do not confuse technical bias with ethical or social bias.
- Do not answer every question with privacy and security. Consider the environment, society and mental health too.
- Do not treat ethics as fixed. It must be reassessed.
Specific case studies make your answers stronger, so learn two or three well.
A4.4.1 Ethical implications
| Issue | The question | Case study |
|---|---|---|
| Accountability | Who is responsible for a system’s decision: the maker, the user or both? | In 2018 a self-driving test car hit and killed a pedestrian. The safety driver later pleaded guilty to endangerment. |
| Fairness and bias | Models can repeat bias from data or design, so it must be actively found and reduced. | Amazon dropped a recruiting tool that was biased against women. |
| Consent | Training data is often collected without clear permission. | A UK hospital trust shared patient records with DeepMind without properly informing patients. |
| Environmental impact | Training and running models uses a lot of energy, and so has a carbon cost. | Training a single large language model can emit hundreds of tonnes of carbon. |
| Privacy | Models can infer facts that nobody supplied, such as a health condition. | Strava’s global heat map revealed the routes of people on military bases. |
| Security | Attacks include data poisoning, model evasion and model inversion. | Users manipulated Microsoft’s Tay chatbot into posting offensive messages within a day. |
| Societal impact | Jobs, public opinion and civil liberties can all change. | Clearview AI scraped billions of photos for facial recognition. |
| Transparency | Neural networks are hard to explain, even for their makers. | The Apple Card was questioned when a couple with shared assets received very different limits. |
✏️ Book corrections
- The book says DeepMind was found in breach of UK privacy law. The 2017 ruling by the UK Information Commissioner’s Office was against the Royal Free NHS Trust, which supplied the patient data. Consent is still a fair lesson to draw.
- On COMPAS, the book states a racial bias as simple fact. A 2016 ProPublica analysis found that Black defendants were more often wrongly flagged as high risk, while the makers disputed the method. This debate is useful: different measures of fairness can conflict.
A4.4.1 Machine learning in online communication
| Biased training data | Over- or under-represented groups affect predictions and reliability. |
| Misinformation | Models can generate believable false text, images and video cheaply. |
| Echo chambers | Recommendation systems chase engagement, showing more of the same and fewer other views. |
| Online harassment | Bots and deepfakes can scale up abuse, and authorities struggle to keep pace. |
| Privacy and anonymity | Users may believe they are anonymous when data can be re-identified. |
In 2006, Netflix released 100 million “anonymous” film ratings. Researchers matched them to public reviews on another site, and re-identified users. Try the same idea below.
A4.4.2 Reassessing ethics
As technology joins daily life, new questions keep arising, so guidelines need regular review.
| Quantum computing | May break today’s cryptography, so quantum-resistant methods are needed. |
| Augmented reality | Collects data about surroundings, and alters what people perceive. |
| Virtual reality | Raises questions about mental health, escapism and limits on content. |
| Pervasive AI | How do we guard against intrusive surveillance and constant data collection? |
| Privacy | Who owns data about you: you or the company that collected it? |
| Equity | Will technology narrow or widen gaps between groups and regions? |
Class debate prompts
- Should insurers see predictions of future illness when setting prices?
- Should a self-driving car prioritize pedestrians or its passengers?
- Should a city use facial recognition on CCTV to reduce crime?
- Should platforms be responsible for breaking echo chambers?
Quick quiz
Exam-style questions
These are written for this site in the style of Paper 1. Answer on paper first, then open the mark points.
-
Outline three ethical implications of facial recognition in public spaces. [3]
Mark points
- Privacy: people are identified and tracked without consent [1].
- Bias: accuracy may vary between groups, leading to false accusations [1].
- Accountability: it may be unclear who answers for a wrong match [1].
-
A social media company personalizes newsfeeds. Outline two ethical issues and two strategies to address them. [4]
Mark points
- Echo chambers limit exposure to other viewpoints [1].
- Bias may promote certain groups’ content over others [1].
- Audit the algorithm regularly for bias [1].
- Publish how ranking works, or let users control it [1].
-
A university uses AI for admissions. Describe two possible biases and a measure for each. [4]
Mark points
- Historic data may favour students from certain schools [1]. Remove or reweight such features, and audit outcomes [1].
- Language in essays may disadvantage non-native speakers [1]. Keep a human reviewer in the decision [1].
-
Distinguish between technical bias and ethical bias. [2]
Mark points
- Technical bias is a deviation in the algorithm that reduces accuracy [1].
- Ethical bias is prejudice in the data that leads to unfair outcomes for groups [1].
-
Describe the ethical implications of false negatives in a disaster prediction model. [3]
Mark points
- A missed disaster means no warning is given [1].
- This can cost lives and property [1].
- Lowering this risk raises false alarms, which can erode public trust [1].