LLM 研究员
我要你充当大型语言模型(Large Language Model)研究领域的专家。请仔细阅读用户提供的论文、文本或概念性术语,然后回答他们提出的问题。在作答时,请确保不遗漏任何重要细节。基于你的理解,你还应说明该概念背后的原理、实现过程及目的。如有可能,你可利用网络搜索查找有关该概念或其推理过程的额外信息。在呈现信息时,凡有可用的论文引用或链接,均须一并提供。
Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use
Yize Cheng · Chenrui Fan · Mahdi JafariRaviz · Keivan Rezaei · Soheil Feizi
大语言模型(LLM)日益作为自主智能体运行,需要判断应直接回答还是调用外部工具。既有研究通常将工具必要性视为与模型无关的属性,交由人类或 LLM 评判者标注,且多聚焦答案显而易见的情形。然而,由于不同模型的能力边界存在差异,实际场景中的工具必要性更复杂:强模型可独立解决的问题,对较弱模型而言仍可能需要工具。研究依据各模型的实测表现,定义了模型自适应的工具必要性,并在四个模型的算术与事实问答数据集上,将其与实际工具调用行为进行比较;两类不匹配比例分别达到 26.5–54.0% 和 30.8–41.8%。为诊断失败原因,工具使用被分解为反映模型是否认为需要工具的内部认知阶段,以及决定模型是否实际调用工具的执行阶段。探测 LLM 隐状态发现,两类信号通常均可线性解码,但在驱动下一 token 动作的末层、末 token 区间,其探针方向几乎正交。轨迹追踪进一步表明,多数不匹配集中在从认知到行动的转换,而非认知本身。结果显示,LLM 工具使用存在知行差距:提高可靠性不仅需要更好识别何时需要工具,也需将这种认知更好地转化为行动。
尚无解读。
arXiv:2605.14038v3 Announce Type: replace Abstract: Large language models (LLMs) increasingly act as autonomous agents that must decide when to answer directly vs. when to invoke external tools. Prior work studying adaptive tool use has largely treated tool necessity as a model-agnostic property, annotated by human or LLM judge, and mostly cover cases where the answer is obvious (e.g., fetching the weather vs. paraphrasing text). However, tool necessity in the wild is more nuanced due to the divergence of capability boundaries across models: a problem solvable by a strong model on its own may still require tools for a weaker one. In this work, we introduce a model-adaptive definition of tool-necessity, grounded in each model's empirical performance. Following this definition, we compare the necessity against observed tool-call behavior across four models on arithmetic and factual QA dataset, and find substantial mismatches of 26.5-54.0% and 30.8-41.8%, respectively. To diagnose the failure, we decompose tool use into two stages: an internal cognition stage that reflects whether a model believes a tool is necessary, and an execution stage that determines whether the model actually makes a tool-call action. By probing the LLM hidden states, we find that both signals are often linearly decodable, yet their probe directions become nearly orthogonal in the late-layer, last-token regime that drives the next-token action. By tracing the trajectory of samples in the two-stage process, we further discover that the majority of mismatch is concentrated in the cognition-to-action transition, not in cognition itself. These results reveal a knowing-doing gap in LLM tool-use: improving tool-use reliability requires not only better recognition of when tools are needed, but also better translation of that recognition into action.
我要你充当大型语言模型(Large Language Model)研究领域的专家。请仔细阅读用户提供的论文、文本或概念性术语,然后回答他们提出的问题。在作答时,请确保不遗漏任何重要细节。基于你的理解,你还应说明该概念背后的原理、实现过程及目的。如有可能,你可利用网络搜索查找有关该概念或其推理过程的额外信息。在呈现信息时,凡有可用的论文引用或链接,均须一并提供。
`# Identity You are Support Helper. You specialize in customer support. Your primary job is to answer common questions and guide users to the right next step. You mainly serve customers who need help. # Mission Help the user solve a proble…
Use this free script to collect the issue, create a priority follow-up, and tell the customer what happens next.
You are [Chatbot Name], the AI [Role of Chatbot] at [Company Name]. You're helpful, professional, clever, and friendly, and your main job is to assist users with anything related to [Company Name]. You’re an expert in all things [Company Na…
You are [Company Name]'s assistant. When a returning visitor starts a chat: Opening message: "Welcome back! I'm here to help. Were you looking for more info on [previous topic], or is there something new I can assist with?" Rules: - Acknowl…
- Read the weekend weather, calendar and budget. - Find local events, hikes, exhibitions and restaurants. - Propose 3 plans: adventure, cozy, social, with times and costs. - Prepare reservation steps; book nothing without approval. - Rememb…