cs.RO提交于 2026-10-26已译

ArtifactArena：以模型在物理世界中所构建之物进行评估

ArtifactArena: Evaluating Models by What They Build in the Physical World

Kushagra Tiwary · David Mayo · Nikhil Behari · Xiangzhou Sun · Abdulrahman Alabdulkareem · Isaac Galatzer-Levy · et al.

中文摘要

评估前沿模型时，应关注其能否在具身物理环境中设计并制造物体，而非只衡量语言表达。ArtifactArena 是一个开放式平台，模型需完成软硬件协同设计，在模拟竞技场中构建可完全运行的机器人并展开竞赛。三种评测机制分别结合文本描述、物理模拟器反馈和比赛数据，对机器人产物进行零样本生成、验证器引导改进及开放式物理设计。各项机器人的正面对抗结果用于生成前沿模型的 Elo 排名。框架与赛事基础设施将持续开放社区提交，形成可动态更新且不会饱和的测试平台，用于衡量开放式智能在物理世界中的能力边界。

关键要点

01提出 ArtifactArena 开放式平台，评测模型在物理环境中的工程与创造能力。
02任务要求模型协同设计软硬件，构建能在模拟竞技场中竞赛的完整机器人。
03通过文本描述、物理模拟器反馈和比赛数据，支持零样本与迭代改进评测。
04依据机器人产物的正面对抗赛事计算前沿模型的 Elo 排名。
05持续接受社区提交，以建立可扩展且非饱和的开放式智能测试平台。

解读

尚无解读。

原始英文摘要

arXiv:2610.06511v2 Announce Type: replace Abstract: To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce ArtifactArena, an open-ended platform where models face a physically grounded hardware-software co-design challenge: engineering fully functional robots to compete in a simulated arena. We evaluate a frontier model's zero-shot, verifier guided refinement, and open-ended physical design capabilities through three harnesses that refine their bot artifacts based on text descriptions, physics simulator feedback, and gameplay data. We benchmark these capabilities with an Elo ranking of frontier models derived from head-to-head tournaments between their artifacts. By releasing this framework and tournament infrastructure for ongoing community submissions, we establish a living, non-saturating testbed to continuously measure the expanding limits of open-ended intelligence in the physical world. Please visit https://artifactarena.ai for more information.