跳到正文
返回论文列表
cs.RO提交于 已译

GAWM:用于多任务机器人操作的紧凑潜在世界-动作模型

GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation

Yang Liu · Xiaoli Liu · Liuzhao Li · Yuduo Song · Muyuan Liu · Xiang Gao

中文摘要

世界-动作模型(WAMs)将未来观测预测融入机器人策略学习,但现有模型多基于十亿参数视觉语言或视频生成骨干,导致训练成本高且推理延迟大。GAWM(几何感知世界模型)是一种紧凑的多任务WAM,在冻结的DINOv3编码器潜在空间中预测未来观测。它将每个相机视图的特征压缩为少量token,由轻量级预测器根据任务(而非动作)预测两个未来时间步的token,并通过未来帧的密集patch特征进行监督,以编码未来场景的空间布局。动作解码器直接从当前及预测token回归未来动作块。GAWM参数量少于1亿,无需机器人或视频预训练,在RoboTwin 2.0上优于十亿参数WAM,在LIBERO上表现与十亿参数VLA相当。在双臂机器人上,其留出演示集的动作预测误差低于微调后的十亿参数VLA,且推理延迟显著更低。消融实验表明,在训练数据或时间受限时,潜在世界模型能显著提升性能。

关键要点

  1. 01现有世界-动作模型依赖十亿参数骨干,导致训练成本高且推理延迟大。
  2. 02GAWM在冻结DINOv3潜在空间预测未来观测,参数量少于1亿且无需预训练。
  3. 03在RoboTwin 2.0上优于十亿参数WAM,在LIBERO上表现与十亿参数VLA相当。
  4. 04双臂机器人实验中,动作预测误差低于微调VLA,且推理延迟显著更低。
  5. 05消融实验显示,在训练数据或时间受限时,潜在世界模型能显著提升性能。

解读

尚无解读。

原始英文摘要

arXiv:2610.37091v1 Announce Type: new Abstract: World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.

同方向论文 · cs.RO

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

Hugging Face Blog82

LeRobot v0.6.0 发布:引入世界模型策略、奖励模型 API 和新基准

LeRobot v0.6.0 发布,带来世界模型策略(VLA-JEPA、LingBot-VA、FastWAM)让机器人学会想象未来,以及一批新 VLA 模型(GR00T N1.7、MolmoAct2、EO-1、EVO1)。

推荐理由:原文给出了 world model policies、新 VLA 模型、reward models API 等具体能力变化,读者可以据此了解机器人学习工具链的最新进展。