超越下一步:世界模型引导的多智能体场景类人导航
Seeing Beyond the Next Step: World-Model-Guided Human-Like Navigation in Multi-Agent Scenes
Seong-Eun Hong · JuYeong Hwang · Jinhyun Kim · Hanyoung Jang · HyeongYeop Kang
中文摘要
在人群密集场景中选择下一步动作,不只是避开当前障碍:某个动作局部可行,却可能偏离人行横道走上车道、切入过街人流,或堵住狭窄通道。这些失效模式仅凭当前状态无法察觉,需要推理每个候选动作的后果。世界模型恰好提供这种动作条件下的未来预测,但每一步决策都展开推演,部署级导航系统无法承担其开销,领域内大多以速度换取了前瞻性。本文认为这是伪两难。世界模型在人类感知导航中的自然位置是训练时作为偏好预言机,将前瞻蒸馏进快速策略,而非推理时作为在线规划器。方法将离散视觉动作先验与两个任务对齐的世界模型配对,一个建模近邻社会动态,一个建模未来场景语义,按预测后果为候选动作打分,并通过组相对偏好蒸馏精炼策略。部署时两个世界模型均移除,策略单次前向完成动作。在五个留出评估场景中,导航在安全性、社会与群体一致性、目标导向上均优于强反应式与学习基线,朴素观察者评价其运动更类人。结论指向将世界模型前瞻引入可部署智能体的另一条路径:不加速推演,而是将其摊销进策略,使前瞻成本仅限训练阶段。项目页:https://seoneun.github.io/-NeurIPS-2026-Seeing-Beyond-the-Next-Step/
关键要点
- 01问题:仅凭当前状态无法察觉局部可行但会偏离人行横道、切入人流或堵塞通道的动作,需推理候选动作后果。
- 02方法:训练时用两个任务对齐世界模型(近邻社会动态、未来场景语义)作为偏好预言机,通过组相对偏好蒸馏将前瞻注入离散视觉动作先验策略。
- 03结果:部署时移除世界模型,策略单次前向推理;在五个留出场景中安全性、社会与群体一致性、目标导向均优于强反应式与学习基线。
- 04评价:朴素观察者认为生成运动更类人。
- 05局限:前瞻成本被限制在训练阶段,部署时不再在线推演未来。
解读
尚无解读。
原始英文摘要
arXiv:2611.07730v1 Announce Type: cross Abstract: Selecting a next motion in a populated scene is not just a matter of avoiding what is currently in the way: an action can be locally feasible yet drift onto the road instead of the crosswalk, cut into a stream of crossing pedestrians, or bottleneck a narrow passage. These failure modes are invisible from the current state alone, and acting on them requires reasoning about what each candidate action will cause. World models offer exactly this, namely action-conditioned predictions of the future, but rolling them out at every decision step is a cost deployable navigation systems cannot pay, and the field has largely traded foresight for speed. We argue this is a false dichotomy. The natural place for a world model in human-aware navigation is at training time, as a preference oracle that distills foresight into a fast policy, rather than at inference time, as an online planner. We pair a discrete visual action prior with two task-aligned world models, one for nearby social dynamics and one for future scene semantics, that score candidate actions by their predicted consequences and refine the policy through group-relative preference distillation. At deployment, both world models are removed and the policy acts in a single forward pass. Across five held-out evaluation scenarios, this yields navigation that is simultaneously safer, more socially and group-coherent, and more goal-directed than strong reactive and learned baselines, with naive observers rating the resulting motion as more human-like. Taken together, our findings point to a different approach for bringing world-model foresight into deployable agents: not accelerating rollouts, but amortizing them into the policy so that the cost of looking ahead is confined to training. Project page: https://seoneun.github.io/-NeurIPS-2026-Seeing-Beyond-the-Next-Step/