英伟达发布机器人基础模型GR00T 2,模型与数据将陆续开源
英伟达在CoRL大会上发布新一代世界动作模型GR00T 2,可同时预测未来画面与机器人动作,模型、数据、仿真环境和数据采集硬件将陆续开源,10万小时人类数据已率先开放。
推荐理由:英伟达GR00T 2的跨本体迁移路径与数据金字塔构成,可供读者判断机器人基础模型的训练范式变化。
E0.5: From Reconstructive to Future-Predictive 3D Foundations for Robotic World-Action Modeling
Era AI Lab Team · Jianhao Lai · Zhihao Zhan · Zhiquan Liao · Qichang Li · Shubin Chen · et al.
世界-动作模型(WAMs)为具身智能提供了框架,但主流方法仍以视频或视觉语言模型为核心,3D几何特征仅作为辅助。E0.5构建于3D基础模型之上,将3D几何作为世界建模与动作预测的原生表示。该模型显式预测未来3D状态与动作,捕捉几何世界动态。在RoboTwin 2.0基准上,E0.5无需具身数据预训练,在干净与随机化设置下成功率分别为89.90%和50.09%。基于AgiBotWorld-Beta中超过16万条机器人轨迹进行具身预训练后,在AgileX Piper和Zerith H1上的平均真实操作成功率分别提升约15和7个百分点。这些结果展示了几何原生基础模型在获取机器人交互知识并迁移至物理操作中的潜力。
尚无解读。
arXiv:2610.36011v1 Announce Type: new Abstract: World-action models (WAMs) offer a promising framework for embodied intelligence, yet mainstream approaches remain centered on video foundation models or vision-language models. Even when geometric features extracted by 3D models such as VGGT are incorporated, they are typically passed into a video model or VLM that serves as the primary backbone, leaving 3D geometry as an auxiliary input or prediction target rather than the native foundation for world modeling. We present E0.5, a geometry-native world-action model built directly on 3D foundation models. E0.5 introduces two key innovations. First, it adopts a 3D foundation model as the primary backbone, making 3D geometry the native representation for both world modeling and action prediction. Second, it explicitly predicts future 3D states together with actions, enabling the model to capture geometric world dynamics beyond action-only modeling or future video prediction. Experiments on RoboTwin 2.0 first demonstrate the effectiveness of E0.5 without embodied-data pretraining, achieving success rates of 89.90\% and 50.09\% under clean and randomized settings, respectively. Building on these results, we further pretrain E0.5 on more than 160k robot trajectories from AgiBotWorld-Beta, jointly learning robot actions and future geometry within the pretrained 3D foundation. This embodied pretraining improves mean real-world manipulation success rates by approximately 15 and 7 percentage points on AgileX Piper and Zerith H1, respectively. Together, these results demonstrate the potential of geometry-native foundations to acquire robot interaction knowledge and transfer it to physical manipulation. E0.5 embodies this geometry-native approach as a member of the VWA family of world-action models.
英伟达在CoRL大会上发布新一代世界动作模型GR00T 2,可同时预测未来画面与机器人动作,模型、数据、仿真环境和数据采集硬件将陆续开源,10万小时人类数据已率先开放。
推荐理由:英伟达GR00T 2的跨本体迁移路径与数据金字塔构成,可供读者判断机器人基础模型的训练范式变化。