The decision was automated. The bias was not.
Also known as Automated bias / predictive inequality
A system producing outcomes that systematically disadvantage a group, not because anyone programmed prejudice, but because the system learned from a world that already contains it. The model is trained on historical data, and the historical data carries historical unfairness: who was hired, who was lent to, who was policed. The system reproduces the pattern and calls it prediction. The automation gives the bias a veneer of objectivity, because a number feels neutral in a way a person does not. The discrimination is laundered through math, and the math is offered as the reason no one is responsible.
Truth-adjacency
Truth-independent: the pattern works regardless of whether the claim is true
Where it shows up
Platforms and algorithms
The phrases and tells that mark this pattern in the wild:
an automated decision that falls harder on one group than anothera model trained on historical outcomes being used to decide future onesthe explanation 'the algorithm decided' offered where a human once wouldaccuracy overall masking large errors for a specific groupno one able to say why a particular person was rejectedThe tell is the inheritance. Algorithmic discrimination rarely comes from a rule that says to treat a group badly. It comes from a model trained on the past being asked to decide the future, and the past was not fair. Watch for systems that predict outcomes from historical outcomes, and for accuracy reported in the aggregate that hides large errors for a specific group. Also watch for the transfer of authority. When a decision moves from a person to a model, it gains the appearance of neutrality, and that appearance is what lets the bias pass unchallenged. If no one can explain why a particular person was rejected, and the rejection falls predictably on one group, the math is not neutral. It is the past, automated.
You call “algorithmic discrimination” on an unequal outcome that reflects a real, relevant, and fairly measured distinction. A disparity is not always bias. The pattern requires the disadvantage to come from the system reproducing historical unfairness, with the automation lending false objectivity and the errors concentrated on a group. If the outcome tracks a genuine, relevant signal and the errors are not systematically borne by one group, you are looking at a fair prediction. The discipline is to check where the model learned and who bears its mistakes, not to assume every difference is prejudice.
A system is trained on historical outcomes: who was hired, who was lent to, who was policed. The historical data carries historical unfairness. The system reproduces the pattern and calls it prediction.
Because the system appears objective. The math looks neutral. Challenging the outcome feels like challenging math, which is harder than challenging a person.
Later, people realize the math was not neutral. It was the past, automated. The bias was not invented. It was inherited, and someone chose the training data.
Field notes where this pattern was identified:
How this pattern gets misused
Someone treats any unequal outcome as algorithmic discrimination, including a difference that reflects a real, relevant distinction rather than inherited bias. The term becomes a way to call any statistical disparity proof of prejudice, which makes it harder to identify the cases where the system is genuinely reproducing unfairness. Indiscriminate use flattens the difference between a model that learned a bias and one that measured a real pattern.
What it looks like when you're wrong about it
A system producing different outcomes for different groups because the difference reflects a real, relevant, and fairly measured distinction is predicting, not discriminating. The pattern requires the disadvantage to come from the system learning and reproducing historical unfairness, often with the automation lending false objectivity, and with errors concentrated on a group rather than evenly spread. If the outcome tracks a genuine, relevant signal and the errors are not systematically borne by one group, you are looking at a fair prediction, not inherited bias laundered through math.
Not sure? Describe the situation to someone outside it. If they do not see the pattern, pause before you name it.
The name is designed to spread. The hook is designed to stick. If you recognized something, share the name.