OpenFlyV2：面向空中感知与导航的机载视觉-语言-动作模型
OpenFlyV2: An onboard vision-language-action model for aerial perception and navigation
Junli Liu · Zhongrui You · Chenhui Li · Zhen Li · Yuyang Tian · Qizhen Chen · et al.
中文摘要
人类认知能灵活调用共享的感知基础来理解环境、关注被指称的实体、推理空间关系并采取行动。为空中机器人赋予类似的功能组织仍具挑战，现有系统往往只针对单一能力、依赖机外计算，或无法在资源受限平台上持续闭环运行。OpenFlyV2 是一种统一的机载视觉-语言-动作架构，用共享的视觉-语言表示在单一模型中支持语义理解、语言定位感知、空间推理与连续空中控制。紧凑的动作专家（action expert）将共享多模态上下文转换为短时域飞行航点，运动感知的时间记忆与动作先验降低闭环执行的计算开销。构建了基于仿真的数据集，包含约 70,000 条导航轨迹和 60,000 对视觉-语言指令-响应对。OpenFlyV2 在 NVIDIA Jetson Orin NX 上以 10 Hz 完全机载生成动作块，每块含 12 个航点。它从仿真零样本迁移到真实四旋翼飞行，在无真实空中训练数据的情况下于四个真实环境取得 48% 的导航成功率。大量结果表明，广泛的视觉-语言能力与连续闭环飞行可在资源受限的空中平台上共存于单一模型。代码将公开于 https://github.com/Ideal-ljl/AeroBrain。
关键要点
- 01现有空中机器人系统多专精单一能力、依赖机外计算，难以在资源受限平台持续闭环运行。
- 02用共享视觉-语言表示统一语义理解、语言定位感知、空间推理与连续空中控制。
- 03紧凑动作专家输出短时域航点，时间记忆与动作先验降低闭环计算开销。
- 04在 Jetson Orin NX 上以 10 Hz 机载生成含 12 航点的动作块，仿真到真实零样本迁移。
- 05四个真实环境导航成功率 48%，未使用真实空中训练数据。
解读
尚无解读。
原始英文摘要
arXiv:2610.39287v1 Announce Type: new Abstract: Human cognition flexibly draws on a shared perceptual foundation to understand environments, attend to referenced entities, reason about spatial relationships, and act. Endowing aerial robots with an analogous functional organization remains challenging, as existing systems are often specialized for individual capabilities, dependent on offboard computation, or unable to sustain closed-loop operation on resource-constrained platforms. We present OpenFlyV2, a unified onboard vision-language-action architecture that uses a shared vision-language representation to support semantic understanding, language-grounded perception, spatial reasoning, and continuous aerial control within a single model. A compact action expert converts the shared multimodal context into short-horizon flight waypoints, while motion-aware temporal memory and action priors reduce the computational cost of closed-loop execution. We construct \dataset{}, a simulation-based dataset comprising approximately 70,000 navigation trajectories and 60,000 vision-language instruction--response pairs. OpenFlyV2 generates action chunks, each comprising 12 waypoints, entirely onboard at 10 Hz using an NVIDIA Jetson Orin NX. It transfers zero-shot from simulation to physical quadrotor flight, achieving a 48\% navigation success rate across four real-world environments without real-world aerial training data. Extensive results demonstrate that broad vision-language competence and continuous closed-loop flight can coexist within a single model on a resource-constrained aerial platform. Code will be made publicly available at https://github.com/Ideal-ljl/AeroBrain.