跳到正文
返回论文列表
cs.RO提交于 已译

连接视觉-语言推理与学习控制:野外足式导航

Bridging vision-language reasoning and learned control for legged navigation in the wild

Takahiro Miki · Robin Schmid · Manthan Patel · Turcan Tuna · Julia Richter · Julian Nubert · et al.

中文摘要

在多样化的真实环境中自主导航仍是机器人学最大的挑战之一,既需要鲁棒的低层运动控制以应对复杂地形,也需要高层推理来解读环境线索。ROAM(Robust Open-world Autonomous Mobility)系统使足式机器人能够在复杂的阿尔卑斯与城市环境中完成公里级导航。ROAM 将基于互联网规模数据预训练的视觉-语言模型(Vision-Language Model, VLM)的世界知识与通过强化学习(reinforcement learning, RL)在仿真中学习到的鲁棒运动与局部规划策略相结合。VLM 提供常识推理与场景语义解读,评估风险、识别标志,并生成带导航代价的语义标签。开放词汇分割模块将这些标签转换为稠密的实时风险图,用以调节两个预训练的 RL 策略,分别负责敏捷运动与局部导航,从而有效耦合高层 VLM 推理与低层物理控制。真实世界实验显示,系统能够在多种复杂真实环境中自主导航:机器人以极少干预完成公里级徒步,识别标志以规避危险,并评估环境以找到安全的落脚区域。该工作朝着非结构化环境中系统的可靠部署迈出一步,但也凸显了 VLM 可靠性与空间推理方面仍存在的挑战,表明基础模型是关键组件,但尚不是野外导航的完整解决方案。

关键要点

  1. 01问题:真实野外导航需兼顾低层运动控制与高层环境语义推理,现有系统难以统一。
  2. 02方法:VLM 生成带代价的语义标签,经开放词汇分割转为实时风险图,调节两个 RL 策略。
  3. 03结果:足式机器人完成公里级阿尔卑斯与城市徒步,极少干预,能识标志避险并选安全落脚点。
  4. 04局限:VLM 可靠性与空间推理仍不足,基础模型是关键组件但非完整导航方案。

解读

尚无解读。

原始英文摘要

arXiv:2611.25215v1 Announce Type: new Abstract: Autonomous navigation in diverse real-world environments remains one of the greatest challenges in robotics. It demands both robust low-level motion control to handle diverse terrain and high-level reasoning to interpret environmental cues. We present ROAM (Robust Open-world Autonomous Mobility), a system that enables legged robots to perform kilometer-scale navigation in complex alpine and urban settings. ROAM combines the world knowledge of a Vision-Language Model (VLM), pre-trained on internet-scale data, with robust locomotion and local planning policies learned in simulation via reinforcement learning (RL). The VLM provides common-sense reasoning and semantic interpretation of the scene, assessing risks, interpreting signs, and producing semantic labels with associated navigation costs. An open-vocabulary segmentation module converts these labels into a dense, real-time risk map, which conditions two pre-trained RL policies responsible for agile locomotion and local navigation, effectively coupling high-level VLM reasoning with low-level physical control. Real-world experiments showed the system's ability to autonomously navigate a wide range of complex real-world environments. The robot completes kilometer-long hikes with minimal intervention, interprets signs to avoid danger, and evaluates its environment to identify safe areas on which to step. This work represents a step toward the reliable deployment of systems in unstructured environments. However, it also underscores the challenges that remain in VLM reliability and spatial reasoning, indicating that foundational models are a crucial component but not yet a complete solution for navigation in the wild.

同方向论文 · cs.RO

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 2

Google DeepMind70

Google DeepMind 发布 Gemini Robotics-ER 1.6:增强实体推理赋能机器人任务

Google DeepMind 发布 Gemini Robotics-ER 1.6,这是其首个推理优先模型的重大升级版。该模型增强了空间推理和多视角理解能力,专门用于视觉与空间理解、任务规划和成功检测,并能原生调用工具如 Google Search、视觉语言动作模型或第三方函数。

推荐理由:原文给出了具体性能对比和新增的仪表读取功能,读者可以据此判断该模型在机器人实体推理场景中的具体提升和可用性。

NVIDIA Blog72

NVIDIA Cosmos 扩展世界模型训练、评测与部署工具

NVIDIA Cosmos 新增训练工具、仿真检查点与 Cosmos Data Factory 蓝图,用于构建训练数据、训练和适配世界模型,并完成仿真评测与部署。开发者可微调 GR00T、Alpamayo、QWEN、Pi0.5 等模型;Cosmos 3 NIM 在已验证配置中可实现最高 2 倍推理提速,Metropolis Blueprint 3.3 可在同一 GPU 上监控多 40% 的视频流。

推荐理由:新增工具覆盖训练数据构建、模型训练优化、策略测试与部署,并提供了具体应用效果。