cs.IR提交于 2026-11-02已译

LLM 智能体多工具请求中检索结构的完整性与成本

Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents

Shuyang Zhang

中文摘要

需要调用多个工具才能完成请求的智能体，一旦工具检索器漏掉其中任何一个工具就会失败，但工具检索器大多是在只需单个工具的查询上构建和评估的。在 345 条带专家依赖标签的 AppWorld 查询和覆盖 257 个 MCP 工具的 103 项 MCP-Bench 任务这两个多工具测试平台上，比较六类检索结构的代表方案：扁平词法与稠密检索、LLM 查询分解、LLM 导航层级、先服务器后工具匹配、依赖图，以及训练得到的集合级检索器。测量 completeness@k，即完整所需工具集出现在前 k 个结果中的查询占比，同时记录各结构在检索时产生的 LLM 调用次数、token 数与延迟。对全部 7,961 条 ToolRet 查询的分析表明，聚合完整性混合了两种情形：单目标查询在 10 处的完整性为 .58 至 .67，四目标及以上查询仅为 .08 至 .17，且检索器在单目标查询上的排序在多目标查询上发生反转。在两个测试平台上，未在测试平台训练且每查询至多一次 LLM 调用的结构，在 k = 16 时完成的 AppWorld 查询均不超过 10%。表现更好的两种结构成本不同：在 75 至 90 条域内查询上训练的集合级检索器无需 LLM 调用，在 AppWorld 上达到 .33、在 MCP-Bench 上达到 .58；LLM 导航层级达到 .30 和 .40，但每查询中位需 31 至 38 次 LLM 调用、约五分钟。二者失败模式也不同：训练检索器多把缺失工具排到截断线略下方，层级方法则找到目标 API 后即停止，未收集其前置依赖。在官方 AppWorld 智能体的下游检验中，检索集完整的任务 21 次全部成功，不完整的 39 次中仅成功 2 次。

关键要点

01工具检索器多在单工具查询上评估，多工具请求下完整性显著下降
02六类检索结构在两个多工具测试平台上比较完整性与 LLM 调用、token、延迟成本
03未训练且每查询至多一次 LLM 调用的结构在 k=16 时完成 AppWorld 查询不足 10%
04训练集合级检索器无需 LLM 调用达 .33/.58，LLM 导航层级需约五分钟每查询
05检索集完整时下游任务 21/21 成功，不完整时仅 2/39 成功

解读

尚无解读。

原始英文摘要

arXiv:2610.41905v1 Announce Type: cross Abstract: An agent that must call several tools to fulfill a request fails when the tool retriever omits any one of them, yet tool retrievers are mostly built and evaluated on queries requiring a single tool. We compare representatives of six retrieval structures (flat lexical and dense retrieval, LLM query decomposition, LLM-navigated hierarchies, server-then-tool matching, dependency graphs, and trained set-level retrievers) across two multi-tool testbeds: 345 AppWorld queries with expert dependency labels and 103 MCP-Bench tasks over 257 MCP tools. We measure completeness@k, the share of queries whose entire required tool set appears in the top k, alongside the LLM calls, tokens, and latency that each structure incurs at retrieval time. A motivating analysis on all 7,961 ToolRet queries shows that aggregate completeness mixes two regimes: completeness at 10 is .58 to .67 on single-target queries and .08 to .17 on queries with four or more targets, and the ordering of retrievers on single-target queries reverses on multi-target ones. Across the two testbeds, none of the compared structures that is untrained on the testbed and makes at most one LLM call per query completes more than 10% of AppWorld queries at k = 16. The two structures that do better incur different costs: a set-level retriever trained on 75 to 90 in-domain queries reaches .33 on AppWorld and .58 on MCP-Bench with no LLM calls, while an LLM-navigated hierarchy reaches .30 and .40 with a median of 31 to 38 LLM calls and about five minutes per query. Their failure modes also differ: the trained retriever mostly ranks the missing tools just below the cutoff, whereas the hierarchy finds the target API and stops before collecting its prerequisites. In a downstream check with the official AppWorld agent, tasks with a complete retrieved set succeeded 21 of 21 times, whereas incomplete ones succeeded 2 of 39 times.