Constitutional AI

Full definition

CAI has two phases: (1) Supervised — the model generates responses, critiques them against the constitution, revises them, and is fine-tuned on the revisions; (2) RL from AI Feedback (RLAIF) — a preference model trained on AI-generated comparisons drives a PPO loop. In the 2022 paper the constitution was a short list of natural-language principles chosen for research; the version Anthropic published for Claude in May 2023 drew on the UN Declaration of Human Rights, platform guidelines such as Apple's terms of service, other labs' principles and Anthropic's own early research. In January 2026 Anthropic replaced that list with a long-form constitution that explains the reasoning behind Claude's values. CAI scales oversight by reducing the human-labeling bottleneck.

Why it matters

CAI is the most prominent example of scalable oversight — using AI to help align AI. For buyers, it means model behavior can be steered by editing a written document rather than relabeling thousands of examples, which is materially cheaper and more auditable.

Example

Anthropic publishes Claude's constitution (CC0); an enterprise customer can read it, compare its priorities with their own domain policies, and use it to interpret why the model refuses or complies.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Reinforcement Learning from Human Feedback (RLHF)
  Reinforcement Learning from Human Feedback (RLHF) is a training technique that aligns a language model's outputs with human preferences by training a reward model on human comparisons of candidate responses, then optimizing the LLM against that reward.
• Scalable Oversight
  Scalable oversight is the alignment problem of supervising AI systems on tasks where humans cannot easily check the answer themselves — and the family of techniques being developed to keep oversight tractable as models grow more capable.
• Refusal Training
  Refusal training is the post-training step that teaches a model to decline requests that violate its policies (illegal, harmful, privacy-invasive, etc.) — and, critically, to comply with the much larger set of legitimate requests that superficially resemble refused ones.

More governance & risk terms

• Alignment Tax
  The alignment tax is the performance cost — measured on capability benchmarks — of making a model safer, more honest, or more aligned with human values, relative to the same model trained only for capability.
• Deceptive Alignment
  Deceptive alignment is a hypothetical failure mode in which an AI system behaves aligned during training and evaluation — because it predicts that misbehavior will be corrected — but pursues different goals once it determines it is no longer being supervised.