Constitutional AI
Constitutional AI (CAI) is a training technique, introduced by Anthropic in 2022, in which a model critiques and revises its own outputs against a written set of principles, then learns from those revisions instead of from human-labeled preferences alone.
Full definition
CAI has two phases: (1) Supervised — the model generates responses, critiques them against the constitution, revises them, and is fine-tuned on the revisions; (2) RL from AI Feedback (RLAIF) — a preference model trained on AI-generated comparisons drives a PPO loop. The "constitution" is a public list of principles drawn from the UN Declaration of Human Rights, platform terms of service, and Anthropic's own safety research. CAI scales oversight by reducing the human-labeling bottleneck.
Why it matters
CAI is the most prominent example of scalable oversight — using AI to help align AI. For buyers, it means model behavior can be steered by editing a written document rather than relabeling thousands of examples, which is materially cheaper and more auditable.
Example
Anthropic publishes Claude's constitution; an enterprise customer can read the principles, propose additions for their domain, and trace refusal behavior back to specific clauses.
Related terms
- Reinforcement Learning from Human Feedback (RLHF)Reinforcement Learning from Human Feedback (RLHF) is a training technique that aligns a language model's outputs with human preferences by training a reward model on human comparisons of candidate responses, then optimizing the LLM against that reward.
- Reinforcement Learning from Human Feedback (RLHF)Reinforcement Learning from Human Feedback (RLHF) is a three-stage post-training pipeline: supervised fine-tuning, reward-model training on human preference comparisons, and policy optimization (typically PPO) against that reward model.
- Scalable OversightScalable oversight is the alignment problem of supervising AI systems on tasks where humans cannot easily check the answer themselves — and the family of techniques being developed to keep oversight tractable as models grow more capable.
- Refusal TrainingRefusal training is the post-training step that teaches a model to decline requests that violate its policies (illegal, harmful, privacy-invasive, etc.) — and, critically, to comply with the much larger set of legitimate requests that superficially resemble refused ones.
Source & further reading
Primary source: Bai et al. — "Constitutional AI: Harmlessness from AI Feedback" (Anthropic) (2022).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/nl/glossary/constitutional-ai/.