As AI systems scale to handle complex workflows, LLMs need to use more tools and skills. This puts developers in a bind. They either have to figure out how to choose the right skills for each task or face the sheer cost, latency, and context-window exhaustion of shoving thousands of tools and skills into a single LLM prompt.
To address this, researchers at Alibaba have developed SkillWeaver, a framework that leverages a novel technique called Skill-Aware Decomposition (SAD). SAD uses a feedback loop to choose relevant tools and feed them to the LLM, instead of attempting to choose the right skills in one go.
In experiments, SkillWeaver resulted in up to a 99.9% token reduction and a significant increase in task completion and accuracy compared to naively exposing agents to an entire tool library.
SkillWeaver is useful for real-world AI applications where agents must autonomously orchestrate community-contributed tool ecosystems for multi-step tasks like downloading datasets, transforming information, and creating visual reports.
Skills are a key component of modern LLM agent architectures. As AI agents integrate with massive tool ecosystems (e.g., public MCP registries hosting thousands of community-contributed skills), accurately routing user queries to the right skills becomes a great challenge.
Exposing an entire library to an LLM to find the right tool is inefficient. It quickly overwhelms context limits and consumes hundreds of thousands of tokens.
Most current tool-use frameworks attempt to solve this through API retrieval, documentation matching, or hierarchical structures that treat routing as a single-skill selection or per-step problem.
The current industry standard for agentic loops is the ReAct (”think-act-observe”) framework. In a typical ReAct loop, an AI tackles a problem sequentially: it reasons about what to do, takes an action (like calling a tool), observes the result, and then decides on its next step.
However, the study found that traditional ReAct loops completely failed when faced with enterprise-scale libraries, achieving a 0% Decomposition Accuracy (DA), a strict metric measuring the fraction of queries where the predicted number of sub-tasks exactly matches the ground truth. This failure occurs because real-world queries are inherently compositional.
As Xueping Gao, the paper’s author, explained in comments to TechTalks, “The issue is not that ReAct is inherently broken. It is that standard thought-action loops were not designed for explicit compositional routing across thousands of available capabilities.”
Gao pointed out that as tool registries grow, giving a model more tools doesn’t equate to more capability. Instead, it creates three distinct problems: a context problem (massive token consumption), a decision problem (noisy classification among APIs with similar descriptions but different schemas), and a planning problem (managing sequences and dependencies).
The Alibaba researchers frame the problem of handling complex tasks requiring multiple skills as “Compositional Skill Routing.” Given a complex user prompt and a vast library of tools, an agent must simultaneously figure out three things: how to break the request into a sequence of atomic sub-tasks, how to map each sub-task to the best skill available, and how to compose those skills into an executable plan.
SkillWeaver orchestrates this process through three distinct stages: Decompose, Retrieve, and Compose.
In the first stage, an LLM acts as a task decomposer, breaking the user’s query into a sequence of sub-tasks that each require one skill. Once the sub-tasks are clearly defined, the system uses semantic search to create a shortlist of candidate tools for each sub-task.
In the final stage, a planner evaluates the retrieved candidates based on how well they work together. It checks for compatibility between skills to ensure that the outputs of one tool naturally flow into the inputs of the next. It then builds a final execution plan, represented as a directed acyclic graph (DAG). The DAG maps out dependencies so independent tasks can potentially be executed in parallel.
Consider a user asking an AI agent to “Download the dataset, transform it, and create visual reports”:
In the decompose stage, the decomposer LLM breaks this into three distinct sub-tasks: 1) Download dataset, 2) Transform data, and 3) Create reports.
In the retrieve stage, the system searches the library and finds candidates like “api-client” or “http-fetch” for task one, and “csv-parser” or “etl-pipeline” for task two.
In the compose stage, the system evaluates these options, selects the specific combination of “api-client”, “csv-parser”, and “chart-gen” that are most compatible with one another, and wires them together into a final, ready-to-execute workflow.
One of the key challenges of this pipeline is that LLMs often produce generic step descriptions that don’t match the specific, technical vocabulary of the skills available in the library. The challenge is matching the LLM’s step-generation to the boundaries of the available APIs. Most retrieval systems mistakenly assume the LLM’s initial query is well-formed.
Gao illustrates this granularity challenge: “For example, a model might treat ‘download and parse the file’ as one step, even though the skill library contains separate file-fetching and CSV-parsing skills. It can also make the opposite error and turn one HTTP operation into ‘connect,’ ‘send the request,’ ‘receive the response,’ and ‘parse the response,’ even though one skill covers the entire operation.”
To fix this granularity mismatch, SkillWeaver introduces “Skill-Aware Decomposition” (SAD), a loop that helps the agent find the right level of detail for matching skills to sub-tasks.
SAD begins with an initial decomposition, retrieves a small set of potentially relevant skill hints, and then feeds that retrieved skill vocabulary into a second decomposition pass. “The hints provide evidence about the capability boundaries that actually exist in the system,” Gao said. “The model is therefore not merely asked whether its original answer was correct; it is asked to restructure the problem based on the operational vocabulary available to it.”
To evaluate how SkillWeaver performs in realistic scenarios, the researchers created a custom benchmark called CompSkillBench, consisting of 300 multi-step queries of different difficulty levels. To mirror real-world environments, they used a library of 2,209 real-world skills from public MCPs. These cover 24 functional categories like cloud infrastructure, finance, and databases.
For the core engine, the team primarily used a lightweight 7-billion parameter model (Qwen2.5-7B-Instruct) for task decomposition, paired with a standard semantic search retriever (MiniLM with a FAISS index) to find the tools.
SkillWeaver was evaluated against three main setups:
A brute-force “LLM-Direct” method where all tool names were stuffed into the prompt of a large model.
A vanilla LLM-based decomposition without the SAD loop.
A ReAct-style “think-act-observe” agent loop.
The experiments proved that standard LLM behavior falls short when dealing with large tool libraries, but SAD makes a big difference. In the Vanilla setup, the 7B model was able to correctly decompose the tasks only 51.0% of the time. By activating the SAD feedback loop, accuracy jumped to 67.7%. On “Hard” tasks requiring 4 to 5 distinct skills, SAD improved accuracy by 50%.
The study showed that larger models can actually perform worse when unguided. When the researchers tested a larger 14-billion parameter model in the Vanilla setup, its accuracy plummeted below the 7B model’s accuracy because it tended to “over-decompose” tasks into too many microscopic, unnecessary steps. However, once SAD was introduced, the retrieved tool hints anchored the model back to reality and increased its accuracy. The takeaway is that aligning your agent with the vocabulary of your tools is often more impactful than paying for a larger, more expensive LLM.
Another important takeaway is token savings. The LLM-Direct baseline showed that feeding all your tools into the prompt of a large model doesn’t work. Despite having near-perfect task breakdown capabilities, the massive model managed to retrieve the right tool category only 21.1% of the time when flooded with tool options.
SkillWeaver’s targeted retrieve-and-route approach outperformed this in accuracy while radically reducing token consumption. Exposing all 2,209 tools would consume an estimated 884,000 tokens, but SkillWeaver reduces this to just ~1,160 tokens per query. This is a 99% token reduction at the execution phase. SAD introduces a much smaller, fixed hint context during planning, Gao clarified. “So this should be understood as a reduction in execution-time tool context rather than total system computation,” he added.
This drop is about much more than saving API costs. “The practical benefit goes beyond token cost,” Gao said. “A smaller execution-time toolset means fewer irrelevant choices, fewer opportunities to hallucinate an API call, more stable prompts, and a plan that can be logged and reviewed before execution.”
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.