Self-Consistency

Full definition

Introduced by Wang et al. (Google, 2022), self-consistency improved GSM8K accuracy by ~18 percentage points over single-path chain-of-thought on PaLM-540B. The intuition: complex reasoning has many valid paths; correct answers are reachable from many paths while specific errors are not. Self-consistency is the simplest test-time-compute scaling technique and a precursor to modern reasoning models.

Why it matters

Self-consistency is a near-free accuracy lift on math, logic, and structured-reasoning tasks — provided the cost-per-question budget allows N sampled paths. It is also a useful diagnostic: high disagreement across samples is a strong signal of low-confidence answers.

Example

A tax-prep copilot answers complex deduction questions by sampling 8 chain-of-thought paths; the majority answer is returned, and disagreement above 30% triggers escalation to a human preparer.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Chain-of-Thought (CoT)
  Chain-of-Thought (CoT) is a prompting and training technique in which a language model is encouraged to reason step-by-step before producing its final answer — improving accuracy on math, logic, and multi-hop reasoning in large models (in Wei et al., 2022, GSM8K math accuracy more than doubled for the largest GPT and PaLM models).
• Tree of Thoughts
  Tree of Thoughts (ToT) is a reasoning framework where an LLM explores multiple reasoning branches as a tree, generating intermediate "thoughts," evaluating partial states with the model itself, and backtracking — turning generation into deliberate search.
• Model Evaluation
  Model evaluation is the systematic measurement of an AI model's performance, safety, and behavior across representative tasks — using fixed benchmarks, golden datasets, human ratings, and LLM-as-judge methods — both before and after deployment.
• Prompt Engineering
  Prompt engineering is the practice of designing the inputs to a language model — instructions, examples, role definitions, output formats, and constraints — to reliably produce a desired output without changing the model's weights.

More training & optimization terms

• LoRA (Low-Rank Adaptation)
  LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes a pretrained model's weights and injects small trainable low-rank matrices into selected layers — cutting trainable parameters by up to 10,000x and GPU memory by about 3x with negligible quality loss on most tasks.
• Model Pruning
  Model pruning is the technique of removing unnecessary weights, neurons, or structures from a trained neural network to reduce model size and inference cost — often with minimal quality loss when done carefully.