noun · AI failure
Bias
also called model bias
The AI picks up the leanings in what it learned from. If the writing it studied leans one way, its answers can too.
Systematic skew in a model's outputs that comes from its training data, training process or deployment, producing unfair or distorted results.
Training data reflects the people, languages and viewpoints that wrote it, and tuning reflects the preferences of the people who rated outputs. The model absorbs those patterns and can reproduce skewed defaults, stereotypes or gaps in what it handles well.
Real-life example
You use a model to draft job ads and notice it keeps picking certain words for certain roles. Nobody told it to. It learned those habits from the text it studied. You find the pattern only after reading many drafts side by side, not from any single one.
Where the analogy breaks
-
Bias is not only offensive language. It can be quiet, like default assumptions about who does a job or which names and places are common.
-
A prompt alone rarely removes it. Telling the model to be fair can help a little, but patterns from training are still there.
-
There is no neutral model. The goal is to measure it for your use, reduce what you can, and keep a person in charge of decisions that affect people.