Tool Router

Full definition

Models degrade as the tool count grows past 20–50 — accuracy of tool selection drops, and prompt cost rises. Tool routers solve this by retrieving a small candidate set (semantic search over tool descriptions, hierarchical category routing, or a fine-tuned classifier) and showing the model only those tools. Modern stacks: tool search with deferred tool loading (OpenAI's Responses API for gpt-5.4 and later; Anthropic's Tool Search Tool), MCP servers that expose only a subset of tools, and tool retrieval in LangGraph (langgraph-bigtool).

Why it matters

Any enterprise agent with >30 tools needs a tool router to stay accurate and affordable. Without one, prompt cost grows linearly with tool count and selection accuracy collapses — a common reason agent pilots fail to scale.

Example

An IT-helpdesk agent has 240 tools across systems; a tool router (BGE embedding over tool docstrings) returns the top-8 candidates per query, raising tool-selection accuracy from 51% to 89% and cutting prompt tokens 70%.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• AI Agent / Agentic AI
  An AI agent is an LLM-driven system that, given a goal, plans a sequence of steps, calls tools or APIs, observes the results, and iterates until the goal is reached — going beyond single-turn chat to multi-step autonomous action.
• Function Calling
  Function calling is an LLM capability where the model — given a schema of available functions — emits a structured invocation (function name and JSON arguments) instead of free-form text, letting downstream code execute the call deterministically.
• Semantic Search
  Semantic search is information retrieval that ranks documents by meaning rather than exact-keyword overlap, using vector embeddings of query and documents to measure similarity in a learned semantic space.
• Model Context Protocol (MCP)
  The Model Context Protocol (MCP) is an open standard, introduced by Anthropic in 2024, that lets AI applications connect to external tools, data sources, and services through a common interface — analogous to the Language Server Protocol for editors.

More deployment & operations terms

• Browser-Use Agent
  A browser-use agent is an AI agent that operates a real web browser to complete tasks — clicking, typing, scrolling, and reading rendered DOM or screenshots — in place of using APIs.
• Code-Interpreter Agent
  A code-interpreter agent is an AI agent that writes and executes code (typically Python) in a sandbox to answer questions, run analyses, or transform data — using execution as a tool to escape the limitations of pure text reasoning.