When tools go wrong: antipatterns

Once you give an LLM access to tools, people tend to lose all restraint. Every internal service becomes a tool, every vague future use case gets its own stub, and suddenly you are staring at a 40 function manifest that nobody actually understands. Then you are surprised that the model does random things.

The funny part is that most of this is self inflicted. The model is not confused. You are. You handed it a junk drawer and expected precision.

In this post I will walk through a few recurring antipatterns in tool design for LLMs, and then look at how to test whether the model can actually pick and use those tools in a predictable way. Think of it as a unit test for your tool layer, not for the model itself.

The first one is boring but very common. Someone discovers tools, gets excited, and two sprints later there are separate tools for every micro action.

“create_user”, “update_user_name”, “update_user_email”, “update_user_marketing_preferences”, “list_user_marketing_preferences”, “delete_user_marketing_preferences”.

You get the idea. The schema reads like a SOAP API from 2009.

From the model’s perspective this is a wall of near identical names and overlapping descriptions. You assume it will pick the right one because you lovingly documented each nuance in the description field. In practice the selection turns into pattern matching on prefixes and a lot of guessing.

Symptoms usually look like this: the model frequently calls the wrong flavor of a tool, it chains multiple micro tools instead of one higher level action, and small copy changes in the prompt change which tool it calls.

Under the hood you created an information retrieval problem for the model that has no single clear answer. You asked “which of these 27 nearly identical functions is best” and then you blame the model for not reading your mind.

A more sensible design tends to converge around domain actions, not low level CRUD. One tool for “update_user_profile” with a structured input beats five tools that only differ in one field name. The model can handle optional fields. What it cannot handle well is human indecision turned into a function catalog.

At the opposite end of the spectrum you get the god tool. One function that allegedly can do everything as long as you sacrifice enough structure on the altar of flexibility.

It usually looks something like this.

“execute_action” with parameters like “action_type” as a string, a huge “payload” object that may or may not contain anything, and a couple of flags that nobody fully understands anymore.

Developers defend this with the classic “future extensibility” argument. Why define multiple tools when you can route everything through this one giant multiplexer and just add more cases later.

From the outside it even looks neat. One tool, one endpoint, clean schema. Inside it is just a hand written dispatcher with a growing nest of if statements and a graveyard of unused action types.

For the model this is worse than having multiple tools. The description has to explain twenty different behaviors at once. The parameters are vague, everything is nullable, and there is no clear contract for when a particular field should be used. Tool choice becomes trivial, because there is only one, but argument construction becomes a guessing game.

The end result is that you effectively removed the main advantage of tools: explicit structure that constrains what the model can do and how it should describe intent.

The god tool is popular because it postpones design decisions. You do not have to think about boundaries between actions, so you push that thinking into the model and then complain when it mirrors your confusion.

Another favorite is the baroque JSON schema with deeply nested objects and arrays everywhere.

There is usually some grand explanation for this. Someone wants the tool to match a legacy internal API one to one. Someone wants to cram every possible variant into the same schema without breaking compatibility. Or someone just really likes hierarchical data structures.

The outcome is predictable. You end up with input shapes where a single logical action requires filling fields in three different nested objects, and half of them are conditionally required based on some far away flag.

Humans already struggle with this. You know this because you have written helper builders around these APIs for years. Now you expect an LLM, which works on token sequences and rough patterns, to navigate the same maze without missing anything.

You start seeing tool calls where a sub object is entirely missing, or present but in the wrong place, or fields are mixed between levels. Then you tweak the instructions, write more examples, maybe even fine tune, and the failure mode stubbornly stays.

This is not because the model is lazy. It is because you turned a simple intent into a multi step constraint satisfaction problem over a deeply nested graph of objects.

Flattening the schema even slightly usually has an immediate effect. Two levels of nesting instead of five. Fewer conditionally required fields. Clearer naming for things that conceptually live at the same level. You do not need perfect elegance. You just need to stop torturing the model with your love of XML shaped data.

Not every antipattern comes from the schema. Some come from the English around the schema.

Developers like to write tool descriptions as if the model was a human colleague that can ask clarifying questions. You get phrases like “use this for user operations” or “this tool handles content related tasks”.

Then someone adds another tool with almost the same description but slightly different intent. Over time you end up with three tools that “create content”, two that “manage user data” and one that “handles user operations”.

The model does not understand your org chart. It has no idea that one of these tools is for marketing content and the other for product documentation. It sees three similar looking options and does a glorified fuzzy match on the user request.

If you want the model to pick a particular function, the description has to contain the exact category words that appear in the request or at least very close paraphrases. Anything vaguer turns into roulette, especially when tools are added or removed over time.

Clarity here is mostly about having the courage to be specific. “Create marketing email drafts for end users” beats “create content” every single time. It is boring, yes, but you either accept boring strings or you accept random behavior.

Tools sometimes start innocent and then slowly accumulate side effects that nobody fully documents.

A harmless looking “get_user” suddenly also records last access timestamps, resets notification counters and maybe writes an audit log. All of this made sense in isolation. Together it turns a read operation into a write with a lot of accidental state transitions.

The LLM is then expected to understand these subtleties based on a polite one line description like “fetch user data”. Of course it does not. It happily calls the tool in contexts where those side effects are undesirable, because they are invisible at the interface level.

In traditional code you would catch this with code review or hopefully with tests. In tool design for LLMs the same discipline often disappears. People rely on the fact that the model is “smart” and then quietly ship magical behavior.

If a tool mutates state in any way, that needs to be spelled out. Not in a footnote, not in a wiki nobody reads, but in the description that ships with the tool definition. Otherwise do not act surprised when the model treats it as a pure function and your data ends up in a weird half updated limbo.

Avoiding these antipatterns is not about chasing some grand theory of tool design. It is about basic hygiene.

Keep the tool list short enough that a human can read it in under a minute without rolling their eyes. Group related behavior into one tool when it genuinely belongs together. Split it when the parameters or side effects diverge.

Keep schemas shallow enough that you would be willing to type them by hand once in a while. If you feel the need to write ten helper builders around a tool, that is a strong hint that the model will struggle with it.

Make descriptions concrete and boring. Say exactly what the tool does, for which entities, with which constraints. Mention side effects explicitly. If two tools sound similar when you read their descriptions out loud, they will be confused in practice.

None of this is exciting. It is mostly the same common sense that applies to any public API. The only twist is that here the primary consumer is a language model that cannot console log its way out of a bad interface.

Now to the part that people keep postponing: testing.

If tools are your model’s hands, you probably want to know whether it uses them in vaguely sane ways before you wire them to production systems. Staring at a few manual transcripts is not testing. That is just vibes based QA.

The minimal bar is to treat each tool like a unit in unit testing. For every tool you care about, define a set of input prompts that should lead to a particular tool call with particular arguments. Then run those prompts through your model plus tool layer and assert on the emitted calls.

You do not have to build all of this from scratch. Frameworks like DeepEval give you a harness for defining prompt suites, capturing tool traces and writing assertions around them. You still have to do the thinking about what “correct” looks like, but at least you are not reinventing a test runner and reporting pipeline.

A simple test might look like this:

• Prompt: “Create a marketing email draft for inactive users in Germany about our new pricing.”

• Expected tool call: create_marketing_email with arguments { segment: "inactive", region: "DE", topic: "pricing_update" } and no calls to generic send_notification or update_user tools.

For trickier separation cases:

• Prompt A: “Draft a release note for the new dashboard feature for internal admins.”

• Expected: create_product_changelog.

• Prompt B: “Draft a campaign email for end users about the new dashboard feature.”

• Expected: create_marketing_email.

You can encode these as snapshot tests on the tool invocation trace and let DeepEval (or your own harness) run them on every change.

This does not require anything fancy. You can capture the tool invocation trace, serialize it, and compare against expected snapshots. For some cases you want strict equality on the arguments. For others you allow a bit of fuzziness, for example free form text fields where the exact wording does not matter.

The important part is that these tests run automatically and break loudly when someone adds, removes or changes tools. Yes, this costs money: model calls are not free, and running evaluation suites on CI will show up on your bill. But losing customers because your tool layer silently regressed, or staffing a team of people to click through transcripts all day, is almost always more expensive.

If a new tool appears with a confusing description, you want to see older tests suddenly routing to the wrong function. That is your early warning that tool selection became ambiguous. Without tests you will only notice when a user report eventually surfaces three weeks later.

You can also build small focused suites for trickier cases. For example, if two tools are intentionally similar but should be used in different regions or product areas, write pairs of prompts that differ only in those aspects and assert that the model consistently separates them.

Over time you get a regression suite that describes not only what your tools do, but how the model is supposed to think about them. A library like DeepEval makes it easier to keep those suites runnable and visible, instead of living in a forgotten notebook somewhere.

Is this perfect coverage. Of course not. You will still see surprising behavior in the wild. But at least you have a safety net for the obvious breakage caused by schema churn and enthusiastic refactoring.

More importantly, the act of writing these tests forces you to confront the sloppiness in your tool design. If you struggle to write a clear, unambiguous prompt that should obviously map to one function, that usually means the function boundaries or descriptions are vague.

At that point the problem is no longer the model. It is the mirror it holds up to your API design. And unlike the model, that part is entirely on you.