Differential Privacy

Full definition

Differential privacy is achieved by adding carefully calibrated noise to data, gradients, or query results so that the presence or absence of any single record changes the output by at most an epsilon factor. DP-SGD applies this during model training. Used in production by the US Census 2020, Apple's telemetry, Google's Chrome statistics, and emerging healthcare and finance AI consortia.

Why it matters

Differential privacy turns "we anonymized the data" — which has repeatedly failed under re-identification attacks — into a mathematically defensible privacy claim. As regulators and auditors mature, expect "DP-SGD with epsilon ≤ X" to become a standard contractual requirement for sensitive AI training.

Example

The US Census Bureau used differential privacy to release 2020 Census data products, accepting that small accuracy losses are worth a provable bound on individual re-identification risk — a precedent now informing healthcare and finance AI deployments.

Related terms

• Federated Learning
  Federated learning is a training paradigm where models are trained across many decentralized devices or organizations — phones, hospitals, banks — by exchanging model updates instead of raw data, so sensitive data never leaves its source.
• AI Governance
  AI governance is the framework of policies, roles, controls, and processes an organization uses to ensure its AI systems are lawful, safe, fair, accountable, and aligned with business intent — across the full lifecycle from problem framing to retirement.
• Training Data
  Training data is the curated dataset used to teach a machine-learning model — every pattern the model can recognize, every bias it inherits, and every limit to its accuracy ultimately traces back to this data.
• Synthetic Data
  Synthetic data is artificially generated training data — produced by AI models or simulators rather than collected from the real world — used to augment, replace, or supplement real datasets while preserving privacy or covering rare cases.
• EU AI Act
  The EU AI Act (Regulation (EU) 2024/1689) is the European Union's comprehensive, risk-tiered regulation of AI systems, the world's first horizontal AI law, with obligations phasing in from February 2025 and full general-purpose AI rules applying from August 2025.