The hidden cost of AI agents: wasted tokens
As more teams adopt MCP (Model Context Protocol) to connect LLMs with tools and APIs, a major inefficiency is emerging.
Every MCP server typically injects all tool schemas into the model context on every turn, even if the model only uses one tool. (This started as a LinkedIn post.)
Example:
- 30 tools ≈ ~3,600 tokens per turn
- 25 turns with 120 tools ≈ 362k tokens spent just on schemas
Most of those tokens are never used. That means higher inference cost, slower agents, and context window pressure.
A simple idea: discover tools only when needed
I recently came across mcp2cli, a small but clever tool. Instead of injecting all tool definitions upfront, it:
- Converts any MCP server or OpenAPI spec into a CLI at runtime
- Lets the model discover tools on demand
- Avoids loading full schemas every turn
The result: 96–99% fewer tokens spent on tool schemas.
Why this matters
As we move toward agentic systems with dozens or hundreds of tools, token efficiency becomes a real architecture concern.
Tools like this hint at a broader pattern: the future of AI agents isn’t just better models. It’s better interfaces between models and tools.
If you’re building with MCP, AI coding agents, or tool-based LLM workflows, this project is worth a look.