跳到正文
返回论文列表
cs.AI提交于 已译

GARDEN:通过证据导航为协作多智能体强化学习设计通用自主奖励

GARDEN: Generalist Autonomous Reward Design Through Evidence Navigation for Cooperative Multi-Agent Reinforcement Learning

Dogan Urgun

中文摘要

协作多智能体强化学习(MARL)能让机器人、自动驾驶车辆等智能体团队学习协同行为,但团队学到什么很大程度上取决于奖励函数。仅靠任务成功提供的学习信号太少,中间奖励通常需要手工设计,且环境或团队一变往往就要重做。大语言模型(LLM)可以把奖励函数写成代码来自动化这一过程,但生成的奖励通常只在单一设定下设计和测试。为让奖励可跨设定复用,提出 GARDEN(Generalist Autonomous Reward Design through Evidence Navigation):针对一族相关场景开发一个奖励程序,并原样迁移到设计时未出现的场景。开发过程中,LLM 奖励工程师检查环境代码、训练记录和智能体行为,草稿奖励通过试训练验证,而学习算法和最终奖励选择规则保持固定。在 Overcooked 和 SMAX 基准的未见场景上,两种厨房布局的任务回报比默认奖励分别提升 58.2% 和 52.5%,两个战斗场景的平均胜率从 51.0% 升至 89.9%。在匹配的候选与探测预算下,这超过了固定诊断反馈的 84.2% 和受 Eureka 启发的奖励反思的 79.2%。结果表明,单个自主设计的奖励可有效复用于相关协作场景。

关键要点

  1. 01问题:手工设计中间奖励成本高,环境或团队一变就要重做,LLM 生成的奖励又常局限于单一设定。
  2. 02方法:GARDEN 用 LLM 奖励工程师检查环境代码、训练记录与智能体行为,经试训练迭代出可迁移的奖励程序。
  3. 03结果:Overcooked 两种布局任务回报提升 58.2% 和 52.5%,SMAX 两场景平均胜率从 51.0% 升至 89.9%。
  4. 04对比:匹配预算下优于固定诊断反馈(84.2%)和 Eureka 式奖励反思(79.2%)。
  5. 05局限:学习算法与最终奖励选择规则固定,迁移仅验证于相关场景族。

解读

尚无解读。

原始英文摘要

arXiv:2610.37926v1 Announce Type: new Abstract: Cooperative multi-agent reinforcement learning (MARL) enables teams of agents, such as robots and autonomous vehicles, to learn coordinated behavior, but what such teams learn is largely determined by their reward function. Because task success alone provides too little guidance for learning, intermediate rewards are usually designed by hand, and this effort must often be repeated whenever the environment or the team changes. Large language models (LLMs) can automate the effort by writing reward functions as code, yet the resulting rewards are typically designed and tested in a single setting. To enable their reuse across settings, GARDEN (Generalist Autonomous Reward Design through Evidence Navigation) is proposed, in which one reward program is developed for a family of related scenarios and transferred, unchanged, to scenarios that were not available during its design. During development, environment code, training records, and agent behavior are investigated by an LLM reward engineer, and draft rewards are tested in trial training runs, while the learning algorithm and the rule for selecting the final reward are held fixed. On unseen scenarios of the Overcooked and SMAX benchmarks, task returns were increased by 58.2% and 52.5% over the default rewards on two kitchen layouts, and the mean win rate in two combat scenarios was raised from 51.0% to 89.9%. Under matched candidate and probe budgets, this exceeded the 84.2% and 79.2% obtained with fixed diagnostic feedback and Eureka-inspired reward reflection. These results show that a single autonomously engineered reward can be reused effectively across related cooperative scenarios.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 4

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

hermes agent desktop default system prompt

You are Hermes Agent, an intelligent AI assistant created by Nous Research. You are helpful, knowledgeable, and direct. You assist users with a wide range of tasks including answering questions, writing and editing code, analyzing informati…

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

xcode-mcp (for pi agent)

--- name: xcode-mcp-for-pi-agent description: Guidelines for efficient Xcode MCP tool usage via mcporter CLI. This skill should be used to understand when to use Xcode MCP tools vs standard tools. Xcode MCP consumes many tokens - use only f…

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

VSCode CodeTour Expert Agent

--- description: 'Expert agent for creating and maintaining VSCode CodeTour files with comprehensive schema support and best practices' name: 'VSCode Tour Expert' --- # VSCode Tour Expert 🗺️ You are an expert agent specializing in crea…