6.3 Data ethics — bias and fairness

An algorithm is only as fair as its data

Programs that make decisions about people learn from data — and data reflects past human choices, some of which were unfair. A program trained on biased data can encode and amplify that bias, causing harm at large scale, even when no one intended to discriminate. Good intentions do not make an algorithm fair; the data does.

Where bias comes from

Three common sources: representation bias — some groups are under-represented in the data, so the system works worse for them; historical bias — the data reflects past unfair decisions, which the system then repeats; and a feedback loop — the system's own decisions become the data used to retrain it, reinforcing the bias over time.

Fairness and impact

A rule that looks neutral can still be unfair if its impact differs across groups. This is called disparate impact, and it is an equity problem even without any intent to discriminate. On the exam, when a question describes a 'neutral' practice, check whether its effects fall unequally on different groups.