跳到正文
返回论文列表
cs.AI提交于 已译

终端智能体训练停滞:解谜数据生成与验证挑战

When Terminal-Agent Training Stalls: Demystifying Data Generation and Verification Challenge

Xi Qin · Isabel Kurth · Xin Cui · Elin Park · Alexander Schaefer · Yaad Oren

中文摘要

使用前沿模型如Claude Opus作为元智能体生成终端任务和验证器用于强化学习训练日益普及。然而,可运行的Docker镜像和可执行测试套件并不能保证终端智能体训练的忠实端到端管道。针对这一差距,提出一个元智能体管道,诊断三类失败:基准无效性、测试工具脆弱性和奖励错位。提示重构和上下文扩展将基线可解性提高5.6倍,但9B模型在Claude Opus生成的任务上于20步内平均pass@2达到81.3%后饱和。添加困难任务在未改变训练配置的情况下将平均pass@2降至20.6%,有力证明可解性带是模型特定的。这些发现表明,元智能体可靠性需要将可解性带校准、验证器审计和基础设施错误计作为一流评估标准,而非事后诊断。

关键要点

  1. 01问题:前沿模型生成的终端任务和验证器无法保证端到端强化训练管道的可靠性
  2. 02方法:提出元智能体管道,诊断基准无效性、测试工具脆弱性、奖励错位三类失败
  3. 03结果:提示重构和上下文扩展使基线可解性提升5.6倍,9B模型20步内pass@2饱和于81.3%
  4. 04结果:添加困难任务使pass@2降至20.6%,证明可解性带具有模型特异性
  5. 05建议:元智能体评估需将可解性带校准、验证器审计、基础设施错误计作为核心标准

解读

尚无解读。

原始英文摘要

arXiv:2610.02405v1 Announce Type: new Abstract: Using a frontier model like Claude Opus as a meta-agent to generate terminal tasks and verifiers for RL training is increasingly common. Yet a runnable Docker image and executable test suite do not guarantee a faithful end-to-end pipeline for terminal agent training. We present a meta-agent pipeline motivated by this gap, diagnosing three classes of failure: benchmark invalidity, harness brittleness, and reward misalignment. Prompt redesign and context extension raise baseline solvability 5.6 times, but a 9B model saturates at 81.3% mean pass@2 within 20 steps on Claude Opus-generated tasks. Adding hard tasks reduces mean pass@2 to 20.6% without changing the training configuration, a strong evidence that the solvability band is model-specific. These findings demonstrate that meta-agent reliability requires solvability-band calibration, verifier audits, and infrastructure error accounting as first-class evaluation criteria, not post-hoc diagnost.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

Hugging Face Blog71

Microsoft 与 Hugging Face 用数据库终态奖励后训练 ThinkingBox 智能体

Microsoft 与 Hugging Face 将 ThinkingBox 的数据库终态检查转化为训练奖励,并对 Qwen3.8-27B 进行 50 步 GRPO 微调。

推荐理由:文章把数据库终态检查转化为训练奖励,公开完整训练流程,并同时量化重复成功率、覆盖率与成本变化。

关联提示词 · 6