AI Bias

Full definition

Bias enters at every stage: which problem is chosen, what data is collected, who labels it, what target is optimized, and how the system is deployed. NIST's AI RMF (2023) and the EU AI Act treat bias as a managed risk requiring documentation, testing, and mitigation throughout the lifecycle. Detection uses disparity metrics (e.g., demographic parity, equalized odds), but no single metric captures all dimensions of fairness.

Why it matters

AI bias is now a regulatory, reputational, and litigation risk. The EU AI Act, NYC Local Law 144, and Canada's AIDA all require bias auditing for high-risk AI. Organizations without a documented bias evaluation process for their deployed models are exposed.

Example

In 2018 Amazon scrapped an internal hiring AI that had learned to penalize résumés containing the word "women's" — a bias inherited from a training set dominated by historically male hires.

Related terms

• AI Audit
  An AI audit is a structured, evidence-based examination of an AI system or AI program against defined criteria — covering training data, model, deployment context, monitoring, and governance — performed by an internal team, an external firm, or a regulator.
• AI Governance
  AI governance is the framework of policies, roles, controls, and processes an organization uses to ensure its AI systems are lawful, safe, fair, accountable, and aligned with business intent — across the full lifecycle from problem framing to retirement.
• Explainability (XAI)
  Explainability (XAI) is the property of an AI system whose decisions can be understood by humans — through model-level documentation, prediction-level attributions, and counterfactual explanations — and a regulatory expectation in finance, healthcare, hiring, and other high-stakes domains.
• Training Data
  Training data is the curated dataset used to teach a machine-learning model — every pattern the model can recognize, every bias it inherits, and every limit to its accuracy ultimately traces back to this data.
• EU AI Act
  The EU AI Act (Regulation (EU) 2024/1689) is the European Union's comprehensive, risk-tiered regulation of AI systems, the world's first horizontal AI law, with obligations phasing in from February 2025 and full general-purpose AI rules applying from August 2025.