Tools: Code Is All You Need

I think the GitHub CLI example isn't entirely fair to MCP. Yes, GitHub's CLI is extensively documented online, so of course LLMs will excel at generating code for well-known tools. But MCP shines in different scenarios.

Consider internal company tools or niche APIs with minimal online documentation. Sure, you could dump all the documentation into context for code generation, but that often requires more context than interacting with an MCP tool. More importantly, generated code for unfamiliar APIs is prone to errors so you'd need robust testing and retry mechanisms built in to the process.

With MCP, if the tools are properly designed and receive correct inputs, they work reliably. The LLM doesn't need to figure out API intricacies, authentication flows, or handle edge cases - that's already handled by the MCP server.

So I agree MCP for GitHub is probably overkill but there are many legitimate use cases where pre-built MCP tools make more sense than asking an LLM to reverse-engineer poorly documented or proprietary systems from scratch.

> Sure, you could dump all the documentation into context for code generation, but that often requires more context than interacting with an MCP tool.

MCP works exactly that way: you dump documentation into the context. That's how the LLM knows how to call your tool. Even for custom stuff I noticed that giving the LLM things to work with that it knows (eg: python, javascript, bash) beats it using MCP tool calling, and in some ways it wastes less context.

YMMV, but I found the limit of tools available to be <15 with sonnet4. That's a super low amount. Basically the official playwright MCP alone is enough to fully exhaust your available tool space.

Ive never used that many. The LLM performances collapse/degrade significantly because of too much initial context? It seems like MCP implems updates could easily solve that. Like only injecting relevant servers for the given task based on initial user prompt.