跳到正文
返回论文列表
cs.RO提交于 已译

Verify2Act:评论家引导的潜在世界模型用于验证语言条件操作计划

Verify2Act: Critic-Guided Latent World Models for Verifying Language-Conditioned Manipulation Plans

Chrisantus Eze · Christopher Crick

中文摘要

视觉语言模型(VLM)能提出多步操作计划,但缺乏场景如何变化的基础模型,而可用于检查这些计划的像素级世界模型速度慢且长时程漂移。Verify2Act 是一个模块化验证框架,将语义动作提议与潜在物理验证分离。VLM 提出候选计划,潜在世界模型(V2A-WM)在 DINOv2 特征空间中想象每个计划,双头评论家在执行前检查想象轨迹的时间一致性和目标满足度。V2A-WM 通过交叉注意力动作接地和因果时间历史窗口扩展了近期流匹配世界模型。完整系统部署在低成本机械臂上,世界模型和评论家仅用数字孪生生成的数据训练。在六个语言条件任务中,Verify2Act 达到 83.3% 任务成功率,而无架构改动的潜在世界模型为 61.7%,仅 VLM 为 53.3%,像素扩散世界模型为 46.7%。它验证了执行的每个计划,未出现规划导致的失败:所有失败片段均由低级抓取或放置滑脱引起。在仿真中,Verify2Act 在 CALVIN 上获得最高五任务链成功率,反射式 VLM 调用比像素扩散验证少 30 倍,并在杂乱螺母装配上获得最高成功率。

关键要点

  1. 01问题:VLM 提出操作计划但缺乏场景变化模型,像素级世界模型慢且长时程漂移。
  2. 02方法:VLM 提议计划,潜在世界模型在 DINOv2 特征空间想象,双头评论家验证时间一致性与目标满足。
  3. 03结果:六个语言条件任务达 83.3% 成功率,高于无改动潜在模型 61.7%、仅 VLM 53.3%、像素扩散 46.7%。
  4. 04结果:CALVIN 五任务链成功率最高,反射式 VLM 调用比像素扩散验证少 30 倍。
  5. 05局限:所有失败片段均由低级抓取或放置滑脱引起,非规划失败。

解读

尚无解读。

原始英文摘要

arXiv:2611.25243v1 Announce Type: new Abstract: Vision-Language Models (VLMs) can propose multi-step manipulation plans, but they have no grounded model of how a scene will change, and the pixel-level world models that could check those plans are slow and drift over long horizons. We present Verify2Act, a modular verification framework that separates semantic action proposal from latent physical verification. A VLM proposes candidate plans, a latent world model (V2A-WM) imagines each of them in the DINOv2 feature space, and a dual-head critic checks the imagined rollouts for temporal consistency and goal satisfaction before anything is executed. V2A-WM extends recent flow-matching world models with cross-attention action grounding and a causal temporal history window. We deploy the full system on a low-cost robot arm, training the world model and critic only on data generated in a digital twin. Across six language-conditioned tasks, Verify2Act reaches 83.3\% task success, compared with 61.7\% for a latent world model without our architectural changes, 53.3\% for the VLM alone, and 46.7\% for a pixel-diffusion world model. It verified every plan it executed and incurred no planning-induced failures: all of its failed episodes were caused by low-level grasp or placement slips. In simulation, Verify2Act obtains the highest five-task chain success on CALVIN among the compared baselines with 30$\times$ fewer reflective VLM calls than pixel-diffusion verification, and the highest success on cluttered nut assembly.

同方向论文 · cs.RO

查看全部 →