Why AI Agents Don't Need Your Full API

The easiest thing you can do is make your entire API available to AI agents. However, "by presenting the LLM with fewer, more relevant tools, we reduce confusion, allowing the model to focus better and achieve higher accuracy"1. You see, different API operations can have similar parameters, descriptions, and, from the outside, look alike. When the LLM selects which tool to use, it compares semantic descriptions. If those descriptions are similar between tools, AI agents won't know which ones to use. In addition to that, every exposed API operation requires injecting its full JSON schema (parameters, types, descriptions, and required fields) into the model's prompt. All that leads to an overload of the AI context window, generating high latency and leaving practically no room for whatever you want your user prompt to be. If exposing more operations is bad, then why are so many doing it? Is there a better alternative? Of course. Keep reading to know more.

This article is brought to you with the help of our supporter, Naftiko.

Naftiko is a platform that helps organizations manage their technical assets as business capabilities. It provides a central space where teams can define and discover what their services can actually do. By moving the focus away from raw API operations, Naftiko helps engineering and product teams work better together.

The more you share, the less accurate the results will be. That’s what happens when you overload an LLM’s context window. To understand why, let’s see how LLMs interpret an API and how that interpretation leads to a decision of which tools to use. To begin with, the ability to interpret an API isn’t something that LLMs can do natively. “Tool use,” or “function calling,” depending on who you ask, is the name of the technique. AI agents control the LLM by directing it to generate a structured query that matches the best API operation schema. This way, instead of trying to come up with a direct answer to a question (the user prompt), LLMs generate an API request that the agent will then make to obtain an answer. So, how can the LLM decide which operation is the best, and then generate an API request with the right parameters? It needs to load the machine-readable definition of all the existing operations, and then parse some of its features. It first extracts the names of all the exposed operations. Then, it calculates the semantic meaning of each operation description. After this step, it goes through the input and output definitions and stores them along with each operation name and semantic meaning. You can see that the more operations you expose, the more information the LLM will need to store, right? But the major problem lies ahead, in the step the LLM takes to decide which operation to use. It has to go through all the available operations and find the one that best matches the user prompt. Having a lot of operations easily leads to loss of accuracy in the decision-making process, which eventually leads to making requests to the wrong API operations. Even worse, if the user prompt leads to an orchestration with multiple steps, erring in deciding which operation to use on each step leads to a failed workflow execution. I imagine you’ve been in this situation more than once by now, right? Fortunately, there’s a solution, and it doesn’t require a lot of heavy lifting, just the right combination of tools.

If the problem is that there are too many exposed API operations, then the solution must be to reduce them. And one way to do that is to, somehow, combine multiple operations into individual domain-specific capabilities. Instead of teaching AI agents how to navigate between API operations, you expose entire workflows behind a single request. The trick is to find the workflows that make the most sense and expose those only. And expose them in a way that the AI agent fully understands. The best option right now is to expose them as MCP tools following the best practices in naming, documentation, parameters, output, and error definitions.

So, how do you expose a group of API operations in a way that AI agents can easily consume and execute? Before exposing them, you have to design them. Think about them as capabilities that end users will need. However, end users won’t execute them directly. Instead, they’ll prompt their AI agents, who, in turn, will be the ones calling those workflow operations. Craft the best possible interface you want those AI agents to have access to and define it using a well-known machine-readable format for easy evolution and maintenance. My choice here is to use OpenAPI or something similar, easy to type and understand. Then, behind each one of those composite operations, you define a workflow that implements the capability you’re exposing. You can do that with any open-source or commercial workflow orchestration tool that exposes an endpoint to trigger executions. Arazzo, for instance, is a great way to define how a workflow behaves in a machine-readable way. So, now you have an OpenAPI definition exposing one or several capabilities, where each one points to a workflow. Every time someone calls one of those composite operations, a workflow is triggered, and the result is returned to the caller. With this approach, you can now easily expose the composite operations as MCP tools using open-source solutions such as reShapr or the Infobip OpenAPI MCP framework. If you want a more integrated path, Naftiko makes it easy to wire these workflow definitions directly into agent-ready tools.

Of course, there’s a cost associated with exposing those capabilities. First, you’ll have to invest time up front to analyze those workflows and write all those machine-readable capability definitions. Then you’ll have to maintain the workflow definitions so they don’t get out of sync with any changes happening on the API operations they connect with. And, of course, all the maintenance of executing those workflows and exposing them as MCP tools, for instance, also comes at a cost. Keep in mind that the whole solution will have new infrastructure pieces and will have to be maintained over time.

By now, it’s clear that you have two paths. On one side, you have the option to keep everything as is and incur rising token costs, higher latency, and unpredictable multi-step agent behavior. On the other side, you can make an investment to implement a capability abstraction layer and keep agent behavior under control. It’s up to you to decide which path you want to take now.