跳到正文
返回论文列表
cs.LG提交于 已译

将泛化性从模型转移到建模者:面向部署风格基准的 PHM 大语言模型智能体

Moving Generality from Model to Modeler: LLM Agents for PHM on a Deployment-Style Benchmark

Zhenling Chen · Zhiguo Zeng · Anne Barros

中文摘要

基于大语言模型(LLM)的编码智能体现在能帮助开发许多作为通用方案发布的模型。泛化性是否应转而存在于为每个新任务构建模型的智能体之中?在预测与健康管理(PHM)领域考察该问题,其中每个模型都必须作用于未见过的设备。部署风格基准 NextUnit-Bench 覆盖 24 个公开数据集,划分为 58 个折,留出单元或工况;每个智能体有两小时交付可执行预测器,随后在隐藏单元上运行。第一,前沿智能体优于 AutoML,后者得分不到最佳智能体的一半;在四个数据集上忠实复现已发表方法,30 个中仅 6 个超过最强智能体自身设计;已发表精度在可复现时依赖论文自身约定,而单一约定——限制测试发动机剩余寿命——可使报告误差减半。第二,智能体优势来自针对每台机器数据所做的设计;一旦完成,几百字的纯文本计划承载两个前沿智能体之间的大部分差距,给定较弱智能体的计划,最强智能体降至该智能体水平,常给出相同预测。然而在接触数据之前,智能体执行谁的计划影响甚微,领域工具包似乎也无帮助:最强者的区别在于从数据中学到多少。第三,此类设计只能到达能执行它们的实现者:给定最强智能体的设计,本地可部署的开源权重模型交付频率减半,即使经过轻量化,得分也不高于没有这些设计时。我们认为,泛化性并未消失,而是从模型转移到建模者。

关键要点

  1. 01部署风格基准 NextUnit-Bench 覆盖 24 个公开数据集、58 个折,留出单元或工况测试。
  2. 02前沿智能体得分超过 AutoML 两倍以上,30 个复现方法中仅 6 个超过其自身设计。
  3. 03几百字纯文本计划承载前沿智能体间大部分差距,弱计划可使最强智能体降至同等水平。
  4. 04本地开源权重模型执行最强设计时交付频率减半,轻量化后得分不升。

解读

尚无解读。

原始英文摘要

arXiv:2611.25252v1 Announce Type: new Abstract: Coding agents built on large language models (LLMs) now help develop many models published as general solutions. Should generality instead reside in the agent that builds a model for each new task? We examine this question in prognostics and health management (PHM), where every model must work on unseen equipment. Our deployment-style benchmark, NextUnit-Bench, covers 24 public datasets split into 58 folds that hold out units or operating conditions; each agent has two hours to deliver an executable predictor, then run on the hidden units. First, frontier agents outperform AutoML, which reaches less than half the best agent's score, and faithful implementations of published methods on four datasets, of which only 6 of 30 beat the strongest agent's own design; published accuracies, where reproducible, rest on the papers' own conventions, and a single convention, capping the remaining life of test engines, can halve the reported error. Second, the agents' advantage is a design made on each machine's data; once made, a plain-text plan of a few hundred words carries most of the gap between two frontier agents, and given a weaker agent's plan, the strongest agent falls to that agent's level, often to identical predictions. Before touching the data, however, it matters little whose plan an agent carries out, and domain toolkits appear to help neither: what sets the strongest apart is how much it learns from the data. Third, such designs reach only implementers that can execute them: given the strongest agent's designs, a locally deployable open-weight model delivers half as often, even when they are lightened, and scores no higher than without them. Generality, we argue, is not disappearing but moving, from the model to the modeler.

同方向论文 · cs.LG

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

Hacker News · 首页78

AgentRun 开源发布:为 Agent 加上自改进与持久执行

Parcha Labs 开源 AgentRun,在 Claude Code、Codex、OpenCode、pi 等 Agent harness 之上加入自改进、持久执行和控制面,采用 Apache-2.0 许可。

推荐理由:原文给出自改进、持久执行与控制面的具体机制和成本案例,读者可据此判断它如何接入现有 Agent 工作流。