跳到正文
返回论文列表
cs.LG提交于 已译

你的策略是隐形的 Harness:Agent 接口的决策几何

Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces

Jian Mu · Ruibo Shen · Yao Shu

中文摘要

语言模型 Agent 通过 Harness(提示、工具及控制逻辑)进行后训练与部署。现有评估常假设模型在某一 Harness 下学到的能力可迁移至其他 Harness。在 KL 正则化控制下,最优策略的对数几率恢复其 Harness 对各动作的偏好,表明策略承载了 Harness 的偏好。部署回报等于 Harness 上限(最佳回报)减去策略失配(次优决策损失)。训练仅改变失配,而更换 Harness 同时改变两项。在 MiniWoB++ 中,仅反转观察字段顺序导致成功率从 85.0% 降至 19.2%,但最优决策不变。在 Terminal-Bench Lite 上,使用强模型估算的“上限减失配”能复现准确率排名,而单独指标无法做到。

关键要点

  1. 01指出固定模型比较不同 Harness 时,混淆了 Harness 提供的上限与模型利用效率。
  2. 02证明在 KL 正则化下,策略对数几率反映 Harness 偏好,部署回报由上限与失配决定。
  3. 03MiniWoB++ 实验显示,仅改变观察字段顺序即可使模型成功率从 85.0% 暴跌至 19.2%。
  4. 04Terminal-Bench Lite 验证了“上限减失配”指标比单一指标更能准确预测 Agent 性能排名。

解读

尚无解读。

原始英文摘要

arXiv:2610.37255v1 Announce Type: cross Abstract: Language model agents are post-trained and deployed through a harness, the prompts, tools, and control logic that present a task to the model and execute its outputs. Harnesses are often revised and judged by how an already trained model performs under them, which assumes that what the model learned under one harness carries over to the next. We ask what a policy learns from its harness and what survives when the harness changes. Under KL-regularized control, we show that the log odds of an optimal policy recover how strongly its harness favors each action over the others, so a trained policy carries the preferences of its harness. Deployed return then equals the ceiling of the harness, the best return it offers, minus the mismatch of the policy, the return lost to its own suboptimal decisions. Training changes only the mismatch, while changing the harness moves both terms. Comparing harnesses with a fixed model therefore mixes what each harness offers with how well that model uses it. In MiniWoB++, post-training makes a model's preferences specific to its training harness. Reversing the order of the observation fields leaves every optimal decision unchanged yet lowers a trained model's success from 85.0% to 19.2%. Conversely, every new harness that a fixed-model comparison rejects but that raises the ceiling outperforms the original agent after further training under it, and ceiling and mismatch measured beforehand predict these gains. For coding-agent harnesses on Terminal-Bench Lite, ceiling minus mismatch estimated with a stronger model as a stand-in for the optimum reproduces the accuracy ranking, while either term alone does not. Code is available at https://github.com/2020-qqtcg/HRP.

同方向论文 · cs.LG

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 6

编程github.com/f/awesome-chatgpt-prompts·外部收录·

CLI silently install software on windows

Ask me for the name of the software as your next question. - You are an IT expert technican. I want you to research, verify and then write powershell commands to silently install or update the software on a Windows 10/11 x86_64 computer. …

研究github.com/f/awesome-chatgpt-prompts·外部收录·

Unified Research and Source Analysis Prompt

Unified, High-Precision Research & Analysis Prompt for ChatGPT and Perplexity AI ROLE & BEHAVIOR You are a professional researcher-analyst. Handle inputs as follows: * If the input is a URL/URI: open it fully with your browsing tool (e.g.…

编程github.com/f/awesome-chatgpt-prompts·外部收录·

Vibe Coding with Commands and Skills

Act as a Vibe Coding Expert with built-in /commands and skills. You are proficient in leveraging AI models for coding and UX/UI design tasks, using a variety of tools and frameworks to streamline the development process. Your task is to: -…

编程github.com/f/awesome-chatgpt-prompts·外部收录·

GitHub Stars Fetcher with Agent Browser

# Using Agent Browser to Fetch GitHub Starred Projects ## Objective Use the Agent Browser skill to log into GitHub and retrieve the starred projects of the currently logged-in user, sorted by the number of stars. ## Execution Steps (Follo…

研究github.com/f/awesome-chatgpt-prompts·外部收录·

ISC Class 12th Exam Paper Analyzer and evaluator

Act as an ISC Class 12th Exam Paper Analyzer. You are an expert AI tool designed to assist students in preparing for their exams by analyzing exam papers and generating insightful reports. Your task is to: - Analyze submitted exam papers a…