Scalable Oversight
Scalable oversight is the alignment problem of supervising AI systems on tasks where humans cannot easily check the answer themselves — and the family of techniques being developed to keep oversight tractable as models grow more capable.
Full definition
Approaches include: (1) AI-assisted critique — a second AI flags errors a human can verify; (2) Debate — two AIs argue and a human judges; (3) Recursive reward modeling — break the task into checkable sub-tasks; (4) Weak-to-strong generalization (OpenAI, 2023) — small supervisors successfully aligning larger models; (5) Constitutional AI / RLAIF. The "sandwiching" methodology (Cotra) measures whether non-experts using AI assistance can supervise expert-level tasks.
Why it matters
As enterprises deploy AI in domains where employees cannot easily verify outputs (medical coding, legal contract review, multi-step financial analysis), scalable oversight stops being academic. It is the core question behind every "human in the loop, but the human can't actually check" workflow.
Example
A radiology AI flags potential findings; a junior radiologist with AI critique tools achieves the same review quality as a senior radiologist alone — sandwiching working in practice.
Related terms
- Constitutional AIConstitutional AI (CAI) is a training technique, introduced by Anthropic in 2022, in which a model critiques and revises its own outputs against a written set of principles, then learns from those revisions instead of from human-labeled preferences alone.
- AI Red-TeamingAI red-teaming is the practice of systematically probing an AI system — using adversarial prompts, jailbreak techniques, and edge-case inputs — to surface harmful, unsafe, biased, or policy-violating behaviors before and after deployment.
- AI GovernanceAI governance is the framework of policies, roles, controls, and processes an organization uses to ensure its AI systems are lawful, safe, fair, accountable, and aligned with business intent — across the full lifecycle from problem framing to retirement.
- Mesa-OptimizationMesa-optimization occurs when a learned model is itself an optimizer — a "mesa-optimizer" — pursuing its own internal "mesa-objective" that may diverge from the training (base) objective the gradient descent process was optimizing.
Source & further reading
Primary source: Bowman et al. — "Measuring Progress on Scalable Oversight for Large Language Models" (Anthropic) (2022).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/de/glossary/scalable-oversight/.