fix

The Workaround Was a Bug Report

MCP Rating builds an MCP gateway, and their headline claim used to be that it cut context by 10×. In September they corrected that to about 3×. This week they published a post that goes further: the premise the number rested on is false in the three clients they measured, and in two of them it was already false when they published.

The sentence was: "Every MCP server in your client config injects its entire tool schema into every turn, whether you use it or not." Claude Code, Cursor and Kiro don't do that by default. They give the model the names of your tools and fetch a tool's full definition only when it's needed.

Post 4 measured with the gateway's own harness, which sent every definition. They didn't check a real client.

What changed in the clients

Claude Code calls it tool search. Names and server instructions load at session start; a ToolSearch tool loads definitions on demand. The changelog says version 2.1.7, in January 2026, began deferring MCP tools by default once their definitions exceeded 10% of the context window — and by 2.1.308 it deferred even a single server with 29 tools.

Cursor calls it dynamic context discovery: tool descriptions synced to a folder on disk, with the agent starting from a small static context and reading a tool's file when it needs one. Cursor reports 46.9% fewer total agent tokens in an A/B test.

Kiro doesn't document it, but its agent gets two functions, tool_load and tool_call, and addresses tools by server::tool id.

The standing cost of ten configured servers with 500 tools: 14,853 tokens per request in Claude Code, 4,330 in Cursor, 26,398 characters in Kiro. With Claude Code's tool search off, 183,889. Nine servers with 125 tools cost about 3,000 tokens in Claude Code and under 1,000 in Cursor — most of the ten-server cost is one server with 375 tools, whose names alone add up.

The whole bill

Standing cost is only half of it, because every request re-sends the prompt. MCP Rating ran one task — call list_directory and return exactly what it says — through each setup.

Claude Code, tool search on: three servers took 3 requests and 81,494 prompt tokens; nine took 3 and 87,599; ten took 3 and 122,903. The gateway took 6 requests and 173,063.

Six requests because the gateway does more work: check what's connected, load the gateway's own tools, connect the server (searching its registry in 3 of 10 runs), call the tool through the gateway, answer. Each of those requests re-sends Claude Code's ~26,000-token base prompt.

Over an eight-turn session it doesn't catch up. The gateway uses about 597,000 tokens; three servers configured directly use 475,000 and nine use 510,000. It draws level only against the ten-server setup, whose tenth server has 375 tools, and only from the third turn.

The reason is the best sentence in the post: "The one-time cost isn't one-time." Everything the gateway did in the first turn — the connected-servers check, the registry search, the connect call, and all their results — stays in the conversation, and every later request sends it again.

With tool search off, the order flips: the gateway is lighter than nine servers from turn 2 and lighter than ten from turn 1, because the configured setups pay for every definition on every request. The workaround was always the right answer to the old client behavior. The client changed.

What survives

The gateway's actual argument, they say, was always secondary: a server you add shouldn't get your environment. It starts each server with a short allowlist — PATH, HOME, USER and a few more — plus only the variables you name, so the API keys in your shell aren't there to read. That doesn't depend on how the client loads tools, and it doesn't stop a server reading files as you unless you turn on container isolation. The README says what each level protects.

That's the honest residue: the security property survives, the token claim doesn't.

The retraction is the model

They remove the README's "Why it uses less of your context" section. They add a dated correction to post 4 saying its first sentence wasn't true of Claude Code or Cursor when it was published. They publish the measurement scripts. They list what they don't claim — not a benchmark, not real servers, not dollars, not other clients, not for long — and they state "we are not neutral."

For a vendor post about its own product, that's unusually good practice, and it's why the measurement is credible.

My take

Three things.

They measured the model, not the system. Their harness sent every definition because their mental model said clients did. The real client is the only oracle, and the cost of checking was one afternoon.

A workaround's value is often a bug report about the platform. When the platform fixes the bug, the workaround's remaining value is whatever else it does — here, secret isolation. If you build a product whose pitch is "the platform does X badly," you should assume X gets fixed and ask what's left.

Prompt economics penalize extra steps. In a stateless-prompt architecture, any setup step is re-sent on every turn, so a proxy that adds a hop must beat the direct path by more than the hop costs. That's an argument against a whole class of middle layers, and it's worth modeling before building one.

The gateway may still be worth running for the secrets. But it's no longer a context optimization, and the post says so in the title.

Source: MCP clients now load tool names, not schemas. Our gateway's token pitch didn't survive it. — MCP Rating.

#ai#engineering#benchmarking#agents