Menu

An 89% Accurate Hiring AI Can Still Be Unfair. Here’s Why.
📰

An 89% Accurate Hiring AI Can Still Be Unfair. Here’s Why.

DEV Community: machinelearning·Vinod Kumar·16 days ago
#SSefcOwe
Reading 0:00
15s threshold

High accuracy is not the same as fair outcomes, and confusing the two is one of the costliest mistakes in AI.

Picture this. A company pilots an AI system that screens job applicants and recommends who should advance to the next round. The team runs tests and sees an overall prediction accuracy of 89%. Everyone is pleased.

Then someone slices the results by demographic group. The rate at which the system recommends applicants differs by more than 20% across groups.

The model is “accurate,” so what went wrong?

Accuracy answers a different question

Accuracy measures how often a model’s predictions match the labels it was trained on. It doesn’t ask whether those labels were fair to begin with.

Suppose the training data is a decade of historical hiring decisions. If those decisions systematically favored some groups, a model that learns them well will score high on accuracy precisely because it has learned to imitate the bias.

This is the most likely explanation for the 20% gap.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.