Chain-of-Thought (CoT)

Full definition

CoT was demonstrated by Wei et al. (2022) by simply appending "Let's think step by step" or providing few-shot examples of stepwise reasoning. The 2024-2025 reasoning-model wave (OpenAI o1/o3, DeepSeek R1, Anthropic's extended thinking) extends CoT into post-training: models are trained to spend variable amounts of inference compute on reasoning before answering.

Why it matters

CoT and reasoning models change AI unit economics: paying 5-10× more per inference can deliver 30-60% higher accuracy on hard tasks. For high-stakes workflows (financial analysis, medical reasoning, code generation, legal research) the trade-off is usually worth it.

Example

On the GSM8K math benchmark, GPT-3.5 jumped from 18% accuracy with direct prompting to 56% with chain-of-thought prompting — without any model change.

Related terms

• Prompt Engineering
  Prompt engineering is the practice of designing the inputs to a language model — instructions, examples, role definitions, output formats, and constraints — to reliably produce a desired output without changing the model's weights.
• Large Language Model (LLM)
  A Large Language Model (LLM) is a deep neural network — almost always a transformer — trained on hundreds of billions to trillions of words to predict the next token, and to generate, summarize, translate, or reason over text.
• Inference Cost
  Inference cost is the dollar cost of running a trained AI model in production — per request, per user, or per business outcome — and the operating expense that determines whether an AI feature has positive unit economics at scale.
• Few-shot / Zero-shot Learning
  Few-shot learning is the ability of a model to perform a new task given only a handful of examples at inference time; zero-shot learning is the same idea with no examples — the model relies entirely on its pretraining and the natural-language instruction.