cs.LG提交于 2026-10-16已译

学习引导 LLM 智能体与多智能体系统

Learning to Steer LLM Agents and Multi-Agent Systems

Leyang Shen · Yaxin Gong · Yang Zhang · Chenxiao Fan · Jiakai Wang · Dong Wang · et al.

中文摘要

基于大语言模型（LLM）的智能体和多智能体系统通过长时程工具使用与协作处理复杂任务，但执行过程中易偏离整体目标，导致资源浪费，即目标漂移（goal drift）。该工作提出一种引导智能体（steering agent），实时监督执行并在漂移发生时将系统拉回正轨。引导智能体观察执行上下文，通过低成本快速路径决定何时干预，通过高成本慢速路径决定如何干预。为解锁评估漂移和生成有效干预的能力，采用强化学习联合训练两条路径，结合成对排序目标与组相对策略优化（GRPO）。在信息检索任务上的实验表明，该智能体有效提升了单智能体和多智能体系统的性能与效率。习得的引导智能可泛化到域外任务，迁移至未见过的执行器，并支持无需重训练即可控制干预频率。

关键要点

01指出 LLM 智能体在多步执行中易发生目标漂移，导致资源浪费和结果偏差。
02提出引导智能体，通过低成本快速路径决定干预时机，高成本慢速路径决定干预方式。
03采用强化学习联合训练，结合成对排序目标与 GRPO 优化，提升漂移评估与干预生成能力。
04实验显示该智能体提升单/多智能体系统性能与效率，且可泛化至域外任务及未见执行器。

解读

尚无解读。

原始英文摘要

arXiv:2610.22316v1 Announce Type: new Abstract: Large language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these systems are prone to deviating from the overall goal as execution unfolds, which wastes execution resources and leads to undesired outcomes, a failure termed goal drift. In this work, we propose a steering agent that supervises execution on the fly and steers the system back on track when drift occurs. The steering agent observes the execution context, deciding when to intervene with a low-cost fast path and how to intervene with a high-cost slow path. To unlock its steering intelligence, the ability to assess goal drift and generate effective interventions, we jointly train the two paths with reinforcement learning, combining a pairwise ranking objective with group-relative policy optimization (GRPO). Experiments on information-seeking search tasks show that the steering agent provides effective supervision for both single-agent and multi-agent systems, improving their performance and efficiency. The learned steering intelligence generalizes to out-of-domain tasks, transfers to unseen executors, and supports controllable intervention frequency without retraining.