Bias (in AI)
Systematic errors in AI outputs reflecting prejudices in training data. Can manifest as gender stereotyping, racial assumptions, or cultural insensitivity in generated content.
Why it matters
AI learns from human-created data, so it absorbs the patterns and prejudices baked into that data. This matters because biased systems can quietly disadvantage people in hiring, lending, and other high-stakes decisions, often without anyone noticing. Even for casual use, it's worth knowing that AI reflects its training data's blind spots, so its "neutral" answers can carry assumptions you'd want to question.
A concrete example
If a resume-screening tool was trained mostly on past hires from one background, it may learn to favor that profile and unfairly downrank equally qualified candidates who look different on paper. The AI isn't malicious; it's echoing skewed history. That's why bias matters most wherever AI influences real decisions about people, and why a human should review outputs that affect someone's opportunities.
How to use it
Test on the cases you care about rather than reasoning about it in the abstract. If a system screens candidates, summarises reviews or ranks anything involving people, run it over a set of examples that differ only in the attribute that should not matter and compare the outputs. That takes an afternoon and it is the only way to find out. Keep a human decision-maker in any process that materially affects someone.
The common mistake
Treating bias as a solved problem because a vendor mentions fairness. Bias enters through training data, through the objective a system optimises, and through how its output is used — a claim on a marketing page tells you nothing about how the system behaves on your data.
Related terms
AI Alignment
The research challenge of ensuring AI systems pursue goals that are beneficial to humans. Misaligned AI could technically achieve its objective while causing unintended harm. Alignment research aims to make AI reliably helpful, harmless, and honest.
Guardrails
Safety mechanisms built into AI systems to prevent harmful, biased, or off-topic outputs. Includes content filters, topic restrictions, output validation, and behavioral boundaries that keep AI responses within acceptable limits.
RLHF (Reinforcement Learning from Human Feedback)
A training technique where human evaluators rank AI outputs, and the model learns to produce responses humans prefer. Used to align models like ChatGPT with human values, making them more helpful and less harmful.
AI Detector
A tool that estimates whether a passage of text was generated by an AI model, by measuring statistical properties of the writing rather than by checking any record of its origin.
Constitutional AI
A training approach where AI models are given a set of principles (a 'constitution') and learn to self-critique and revise their outputs to comply with those principles. Reduces reliance on human feedback for safety alignment.
Data Poisoning
A security attack where malicious data is deliberately introduced into AI training sets to manipulate model behavior. Can cause models to produce biased outputs, bypass safety filters, or leak sensitive information.