Tree of Thoughts

Full definition

Yao et al. (Princeton and Google DeepMind, 2023) showed ToT lifting GPT-4's Game of 24 success rate from 4% with chain-of-thought prompting to 74%, and producing more coherent creative writing and better mini-crossword results. The cost is high — many forward passes per problem — making ToT suited to high-value, low-volume reasoning tasks rather than chat. Modern reasoning-tuned models internalize tree-search-like behavior, reducing but not eliminating the need for explicit ToT.

Why it matters

ToT marks the conceptual shift from "LLM as text generator" to "LLM as search operator." For high-stakes reasoning — proofs, plans, code synthesis — explicit search over reasoning trees still outperforms single-path generation on hard problems.

Example

A planning agent uses ToT with breadth-of-3 and depth-of-4 to schedule a 6-warehouse logistics rotation under a fuel-budget constraint, finding a feasible plan single-path CoT failed to produce.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Chain-of-Thought (CoT)
  Chain-of-Thought (CoT) is a prompting and training technique in which a language model is encouraged to reason step-by-step before producing its final answer — improving accuracy on math, logic, and multi-hop reasoning in large models (in Wei et al., 2022, GSM8K math accuracy more than doubled for the largest GPT and PaLM models).
• Self-Consistency
  Self-consistency is a prompting technique that samples multiple chain-of-thought reasoning paths from an LLM at non-zero temperature, then selects the most common final answer by majority vote — improving reasoning accuracy at the cost of more tokens.
• ReAct Pattern
  ReAct (Reasoning + Acting) is a prompting pattern that interleaves reasoning steps ("Thought") with tool actions ("Action") and tool results ("Observation"), letting an LLM iteratively decompose a task, query the world, and self-correct.
• Agentic Workflow
  An agentic workflow is a multi-step AI process where a model plans, takes actions through tools, observes results, and iterates toward a goal — replacing single-shot prompting with a loop that reasons, acts, and self-corrects.

More training & optimization terms

• LoRA (Low-Rank Adaptation)
  LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes a pretrained model's weights and injects small trainable low-rank matrices into selected layers — cutting trainable parameters by up to 10,000x and GPU memory by about 3x with negligible quality loss on most tasks.
• Model Pruning
  Model pruning is the technique of removing unnecessary weights, neurons, or structures from a trained neural network to reduce model size and inference cost — often with minimal quality loss when done carefully.
• Supervised Fine-Tuning (SFT)
  Supervised Fine-Tuning (SFT) is the stage of LLM post-training where a pretrained model is fine-tuned on curated input-output pairs, typically instruction-response demonstrations written by humans.