Self-Consistency
Self-consistency is a prompting technique that samples multiple chain-of-thought reasoning paths from an LLM at non-zero temperature, then selects the most common final answer by majority vote — improving reasoning accuracy at the cost of more tokens.
Full definition
Introduced by Wang et al. (Google, 2022), self-consistency improved GSM8K accuracy by ~18 percentage points over single-path chain-of-thought on PaLM-540B. The intuition: complex reasoning has many valid paths; correct answers are reachable from many paths while specific errors are not. Self-consistency is the simplest test-time-compute scaling technique and a precursor to modern reasoning models.
Why it matters
Self-consistency is a near-free accuracy lift on math, logic, and structured-reasoning tasks — provided the cost-per-question budget allows N sampled paths. It is also a useful diagnostic: high disagreement across samples is a strong signal of low-confidence answers.
Example
A tax-prep copilot answers complex deduction questions by sampling 8 chain-of-thought paths; the majority answer is returned, and disagreement above 30% triggers escalation to a human preparer.
Related terms
- Chain-of-Thought (CoT)Chain-of-Thought (CoT) is a prompting and training technique in which a language model is encouraged to reason step-by-step before producing its final answer — improving accuracy on math, logic, and multi-hop reasoning by 10-40 percentage points on standard benchmarks.
- Tree of ThoughtsTree of Thoughts (ToT) is a reasoning framework where an LLM explores multiple reasoning branches as a tree, generating intermediate "thoughts," evaluating partial states with the model itself, and backtracking — turning generation into deliberate search.
- Model EvaluationModel evaluation is the systematic measurement of an AI model's performance, safety, and behavior across representative tasks — using fixed benchmarks, golden datasets, human ratings, and LLM-as-judge methods — both before and after deployment.
- Prompt EngineeringPrompt engineering is the practice of designing the inputs to a language model — instructions, examples, role definitions, output formats, and constraints — to reliably produce a desired output without changing the model's weights.
Source & further reading
Primary source: Wang et al. — "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (ICLR) (2023).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/ko/glossary/self-consistency/.