Tool Router

Full definition

Models degrade as the tool count grows past 20-50 — accuracy of tool selection drops, and prompt cost rises. Tool routers solve this by retrieving a small candidate set (semantic search over tool descriptions, hierarchical category routing, or a fine-tuned classifier) and showing the model only those tools. Modern stacks: OpenAI Assistants v2 file search + custom routers, Anthropic MCP servers with selective exposure, LangChain ToolRetriever.

Why it matters

Any enterprise agent with >30 tools needs a tool router to stay accurate and affordable. Without one, prompt cost grows linearly with tool count and selection accuracy collapses — a common reason agent pilots fail to scale.

Example

An IT-helpdesk agent has 240 tools across systems; a tool router (BGE embedding over tool docstrings) returns the top-8 candidates per query, raising tool-selection accuracy from 51% to 89% and cutting prompt tokens 70%.

Related terms

• AI Agent / Agentic AI
  An AI agent is an LLM-driven system that, given a goal, plans a sequence of steps, calls tools or APIs, observes the results, and iterates until the goal is reached — going beyond single-turn chat to multi-step autonomous action.
• Function Calling
  Function calling is an LLM capability where the model — given a schema of available functions — emits a structured invocation (function name and JSON arguments) instead of free-form text, letting downstream code execute the call deterministically.
• Semantic Search
  Semantic search is information retrieval that ranks documents by meaning rather than exact-keyword overlap, using vector embeddings of query and documents to measure similarity in a learned semantic space.
• Model Context Protocol (MCP)
  The Model Context Protocol (MCP) is an open standard, introduced by Anthropic in 2024, that lets AI applications connect to external tools, data sources, and services through a common interface — analogous to the Language Server Protocol for editors.