RoboSeed：视觉-语言-动作模型与世界-动作模型的多维评估：能力、资源可负担性与研究证据
RoboSeed: Multidimensional Evaluation of Vision-Language-Action and World-Action Models: Capability, Resource Affordability, and Research Evidence
Xin Ma · Furong Chen · Xinlong Man · Bo Zhang · Yalong Guo · Haolan Lei · et al.
中文摘要
选择机器人动作模型需要关于任务能力、适配成本、部署需求以及所报告结论可靠性的证据。仅凭成功率对不同计算预算和目标的研究者指导有限。提出 RoboSeed，一个面向视觉-语言-动作（VLA）模型与世界-动作模型（WAM）的多维评估框架，属于持续训练、评估并公开释放机器人模型、为模型选择提供可验证证据的计划。框架定义五个维度：能力、下游训练可负担性、部署可负担性、本地验证的研究价值、复现证据。用固定归一化锚点将经验测量与审计标准映射到 0-100 画像，并提供均衡、训练受限、部署受限三种偏好设置。接口与协议检查确定比较何时合适。缺失测量保持显式，不完整的聚合分数不予给出，加权输入覆盖率识别证据缺口。将框架应用于代表性 VLA 与 WAM 模型，依据本地训练、释放权重评估、任务级审计和评估系统研究。结果揭示预算依赖的主干选择、对适配配方的敏感性以及语言与视觉任务接口的差异。可复现评分代码与机器可读证据支持扩展到更多模型。研究提供维度级评估与进一步测量的优先事项，但不完整的服务轨迹与异构协议排除了总体数值排名。
关键要点
- 01仅凭成功率不足以指导不同计算预算下的模型选择，需多维证据。
- 02框架含能力、训练与部署可负担性、研究价值、复现证据五个维度。
- 03用固定归一化锚点生成 0-100 画像，支持三种偏好设置。
- 04结果显示主干选择依赖预算，且对适配配方敏感。
- 05服务轨迹不完整与协议异构导致无法给出总体数值排名。
解读
尚无解读。
原始英文摘要
arXiv:2610.40026v1 Announce Type: new Abstract: Choosing a robot action model requires evidence about task capability, adaptation cost, deployment requirements, and the reliability of reported conclusions. Success rates alone offer limited guidance to researchers with different computing budgets and goals. We propose RoboSeed, a multidimensional evaluation framework for vision-language-action (VLA) and world-action models (WAM), within a continuing program to train, evaluate, and openly release robot models with verifiable evidence for model selection. The framework defines five dimensions: capability, downstream training affordability, deployment affordability, locally verified research value, and reproduction evidence. It maps empirical measurements and audit criteria to 0-100 profiles using fixed normalization anchors, with balanced, training-constrained, and deployment-constrained preference settings. Interface and protocol checks establish when comparisons are appropriate. Missing measurements remain explicit, incomplete aggregate scores are withheld, and weighted input coverage identifies evidence gaps. We apply the framework to representative VLA and WAM models, drawing on local training, released-weight evaluation, task-level audits, and an evaluation-system study. The results reveal budget-dependent backbone choices, sensitivity to adaptation recipes, and differences between language and visual task interfaces. Reproducible scoring code and machine-readable evidence support extension to additional models. The study provides dimension-level assessments and priorities for further measurement, while incomplete serving traces and heterogeneous protocols preclude overall numerical rankings.