Differential Privacy

Full definition

Differential privacy is achieved by adding carefully calibrated noise to data, gradients, or query results so that the presence or absence of any single record changes the output by at most an epsilon factor. DP-SGD applies this during model training. Used in production by the US Census 2020, Apple's telemetry, Google's Chrome statistics, and emerging healthcare and finance AI consortia.

Why it matters

Differential privacy turns "we anonymized the data" — which has repeatedly failed under re-identification attacks — into a mathematically defensible privacy claim. As regulators and auditors mature, expect "DP-SGD with epsilon ≤ X" to become a standard contractual requirement for sensitive AI training.

Example

The US Census Bureau used differential privacy to release 2020 Census data products, accepting that small accuracy losses are worth a provable bound on individual re-identification risk — a precedent now informing healthcare and finance AI deployments.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Federated Learning
  Federated learning is a training paradigm where models are trained across many decentralized devices or organizations — phones, hospitals, banks — by exchanging model updates instead of raw data, so sensitive data never leaves its source.
• AI Governance
  AI governance is the framework of policies, roles, controls, and processes an organization uses to ensure its AI systems are lawful, safe, fair, accountable, and aligned with business intent — across the full lifecycle from problem framing to retirement.
• Training Data
  Training data is the curated dataset used to teach a machine-learning model — every pattern the model can recognize, every bias it inherits, and every limit to its accuracy ultimately traces back to this data.
• Synthetic Data
  Synthetic data is artificially generated training data — produced by AI models or simulators rather than collected from the real world — used to augment, replace, or supplement real datasets while preserving privacy or covering rare cases.
• EU AI Act
  The EU AI Act (Regulation (EU) 2024/1689) is the European Union's comprehensive, risk-tiered regulation of AI systems, the world's first horizontal AI law, with obligations phasing in from February 2025: general-purpose AI model rules from 2 August 2025 (Commission fines from 2 August 2026; models already on the market have until 2 August 2027) and, after the Digital Omnibus (Regulation (EU) 2026/1744), high-risk rules from 2 December 2027 (Annex III) and 2 August 2028 (Annex I).

More governance & risk terms

• Factuality
  Factuality is the property of an AI system's outputs being verifiably true with respect to a trusted reference corpus or world knowledge — a distinct dimension from fluency, helpfulness, or generic accuracy.
• Jailbreak
  A jailbreak is an adversarial prompt or technique that bypasses an AI model's safety guardrails to elicit content or behaviors the model was trained to refuse — for example, hazardous instructions, restricted personal data, or off-policy assertions.
• Prompt Injection
  Prompt injection is an attack in which adversarial text — placed directly in user input or hidden inside content the model retrieves — overrides developer system instructions and hijacks the model's behavior, exfiltrating data or causing unauthorized actions.