Refusal Training

Full definition

Done well, refusal training reduces both true-positive harms and false-positive refusals. Done poorly, it produces models that refuse benign chemistry homework, medical questions, or fiction with violence. Vendors publish behaviour specifications (such as OpenAI's Model Spec) and report over-refusal on public test suites such as XSTest, an academic benchmark (Röttger et al., 2024). Rates have fallen sharply, though they vary by test set: Anthropic reports Claude 2.1 (2023) wrongly refused 35.1% of XSTest's safe prompts versus 9% for Claude 3 Opus (2024), and its May 2025 system card puts Claude Opus 4's over-refusal on benign sensitive-topic prompts at 0.07%, with a 98.4% harmless-response rate on violative requests.

Why it matters

Over-refusal can sink an enterprise pilot. A model that refuses 8% of legitimate queries makes a customer-service automation impossible. Buyers must run domain-specific refusal evaluations, not rely on vendor benchmarks alone.

Example

A health-insurance chatbot built on a 2023-era model refuses 14% of plan-coverage questions citing "medical advice"; switching to a 2025 model with calibrated refusal training drops that to under 1% with no rise in unsafe outputs.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Reinforcement Learning from Human Feedback (RLHF)
  Reinforcement Learning from Human Feedback (RLHF) is a training technique that aligns a language model's outputs with human preferences by training a reward model on human comparisons of candidate responses, then optimizing the LLM against that reward.
• Constitutional AI
  Constitutional AI (CAI) is a training technique, introduced by Anthropic in 2022, in which a model critiques and revises its own outputs against a written set of principles, then learns from those revisions instead of from human-labeled preferences alone.
• Alignment Tax
  The alignment tax is the performance cost — measured on capability benchmarks — of making a model safer, more honest, or more aligned with human values, relative to the same model trained only for capability.
• AI Guardrails
  AI guardrails are the layered controls — input filters, output classifiers, policy engines, schema validation, and structured generation — that constrain an AI system to safe, on-policy, and on-task behavior.

More governance & risk terms

• EU AI Act Risk Tier
  EU AI Act risk tiers are the four-level classification — unacceptable, high, limited, and minimal/no risk — that determines which obligations apply to an AI system placed on the EU market under Regulation (EU) 2024/1689.
• AI Act Conformity Assessment
  A conformity assessment under the EU AI Act is the procedure by which a high-risk AI system is verified to meet the regulation's requirements (Articles 8–15) before being placed on the EU market or put into service.
• Fundamental Rights Impact Assessment (FRIA)
  A Fundamental Rights Impact Assessment (FRIA) is the structured analysis required by Article 27 of the EU AI Act for certain deployers of high-risk AI systems — bodies governed by public law, private entities providing public services, and deployers of credit-scoring AI or of life- and health-insurance risk-assessment and pricing AI (Annex III points 5(b) and 5(c)).