英伟达发布机器人基础模型GR00T 2,模型与数据将陆续开源
英伟达在CoRL大会上发布新一代世界动作模型GR00T 2,可同时预测未来画面与机器人动作,模型、数据、仿真环境和数据采集硬件将陆续开源,10万小时人类数据已率先开放。
推荐理由:英伟达GR00T 2的跨本体迁移路径与数据金字塔构成,可供读者判断机器人基础模型的训练范式变化。
What Does World Modeling Actually Improve in Robot Action Learning?
Hanwen Cui · Yuanyuan Li · Qing Guo · Zhida Zhang · Chao Chen · Xi Chen · et al.
世界动作模型(World Action Models, WAMs)将未来视频预测与机器人动作学习相结合,但仅凭任务成功率提升,无法判断预测监督如何影响执行行为,也无法确认其收益能否在任务和环境变化下持续。研究在 FastWAM 与 OpenWAM 内开展受控的未来视频预测损失消融实验,各模型家族固定架构、初始化、机器人训练数据及训练步数。在已见任务的干净条件下,预测监督分别使成功率提高 24.38 和 72.91 个百分点。轨迹聚类进一步显示,若干粗粒度轨迹簇的成功率上升,即使部分簇的频率变化很小,表明收益并非仅来自轨迹模式占比改变。然而,这些优势未能在分布偏移下稳定保持:未见任务上的改进因模型和任务而异;在所评估的环境随机化下,两种模型成功率均较低,预测监督也未呈现一致优势。研究还将未来视频预测作为 π₀.₅ 的辅助训练目标,仅在训练期间使用轻量级 Transformer 预测器,使成功率从 82.94% 提升至 89.19%。结果表明,未来视频预测可作为动作学习有效的辅助监督,同时区分了熟悉任务上的执行改善与可靠迁移;在所评估环境中,此类收益无需依赖大型预训练视频扩散 Transformer(DiT)。
尚无解读。
arXiv:2610.34593v1 Announce Type: new Abstract: World Action Models (WAMs) combine future-video prediction with robot action learning, yet improvements in task success alone do not reveal how predictive supervision manifests in execution behavior or whether its benefits persist under task and environmental shifts. We conduct controlled future-video prediction loss ablations within FastWAM and OpenWAM, keeping model architecture, initialization, robot training data, and training steps fixed within each model family. Under clean conditions on seen tasks, predictive supervision increases success rates by 24.38 and 72.91 percentage points, respectively. Trajectory clustering further reveals higher success rates within several coarse trajectory clusters, including cases where cluster frequencies change little, indicating that the gains are not solely reflected in changes in the prevalence of trajectory patterns. However, these benefits do not consistently persist under distribution shifts: improvements on unseen tasks vary across models and tasks, while both models exhibit low success rates under the evaluated environmental randomizations, without a consistent advantage from predictive supervision. We further introduce future-video prediction as an auxiliary training objective for $\pi_{0.5}$, where a lightweight Transformer predictor used only during training increases success from 82.94% to 89.19%. These findings support future-video prediction as effective auxiliary supervision for action learning, while distinguishing improved execution on familiar tasks from reliable transfer and showing that such gains need not rely on a large pretrained video diffusion transformer (DiT) in the evaluated setting.
英伟达在CoRL大会上发布新一代世界动作模型GR00T 2,可同时预测未来画面与机器人动作,模型、数据、仿真环境和数据采集硬件将陆续开源,10万小时人类数据已率先开放。
推荐理由:英伟达GR00T 2的跨本体迁移路径与数据金字塔构成,可供读者判断机器人基础模型的训练范式变化。
NVIDIA Cosmos 新增训练工具、仿真检查点与 Cosmos Data Factory 蓝图,用于构建训练数据、训练和适配世界模型,并完成仿真评测与部署。开发者可微调 GR00T、Alpamayo、QWEN、Pi0.5 等模型;Cosmos 3 NIM 在已验证配置中可实现最高 2 倍推理提速,Metropolis Blueprint 3.3 可在同一 GPU 上监控多 40% 的视频流。
推荐理由:新增工具覆盖训练数据构建、模型训练优化、策略测试与部署,并提供了具体应用效果。
LeRobot v0.6.0 发布,带来世界模型策略(VLA-JEPA、LingBot-VA、FastWAM)让机器人学会想象未来,以及一批新 VLA 模型(GR00T N1.7、MolmoAct2、EO-1、EVO1)。
推荐理由:原文给出了 world model policies、新 VLA 模型、reward models API 等具体能力变化,读者可以据此了解机器人学习工具链的最新进展。
Dream Labs 发布 Dream-1 World Action Model,并面向全球开放公共 API,可零样本控制未见过的双臂平行夹爪机器人,无需针对目标机器人微调或提供示范数据。Dream-1 运行时价格为 $3/hour,Dream-1-flash 为 $2/hour;官方也披露了不同机器人上的性能差异,并通过 FaaS 支持基于少量自有数据的适配。
推荐理由:文章同时给出跨机器人零样本控制、公开 API、成本与失败边界,便于评估通用机器人模型的落地条件。