跳到正文
返回论文列表
cs.CV提交于 已译

Wizard:通过智能体控制实时操控任意世界

Wizard: Manipulate Any World in Real Time via Agentic Control

Yuzhe Wu · Zijun Lin · Cheston Tan · Bihan Wen · Yeying Jin

中文摘要

视频世界模型的最新进展使虚拟环境日益逼真且可交互,为玩家自定义玩法开辟了新可能。智能体(agentic)框架通过将显式游戏逻辑与可控世界表示从视觉生成中解耦,进一步推进了这一方向。然而,现有方法仅支持简单游戏设定和预定义运动模式,对未见环境与角色泛化能力差,实时交互性有限。为此,提出 Wizard——一种智能体游戏世界模型,可实时精确控制角色运动,并对新场景、新角色和新动作实现零样本(zero-shot)泛化。构建了覆盖 805 个场景、178 个角色的多样化数据集,引入骨架与几何信息作为结构化代理来条件化视频生成。该智能体框架显式跟踪并更新游戏状态,以支持角色感知的自适应掩码与世界锚定的空间条件化,提升角色身份保持与空间一致性。在 WBench 上的实验表明,该方法在视频质量与时空一致性相当的情况下实现更高的玩法保真度,且参数量与训练数据均少于当前最优世界模型;两步实时变体也保留了这些优势。Wizard 将智能体控制与基于结构化代理的生成相结合,实现玩家驱动的实时、可泛化且机制一致的世界模拟。

关键要点

  1. 01现有智能体世界模型仅支持简单设定与预定义动作,泛化与实时交互不足
  2. 02构建 805 场景、178 角色数据集,用骨架与几何信息作为结构化代理条件化生成
  3. 03显式跟踪游戏状态,实现角色感知自适应掩码与世界锚定空间条件化
  4. 04WBench 上玩法保真度更高,参数量与训练数据少于 SOTA,两步实时变体保持优势

解读

尚无解读。

原始英文摘要

arXiv:2610.40339v2 Announce Type: replace Abstract: Recent advances in video world models have enabled increasingly realistic and interactive virtual environments, opening new possibilities for player-customizable gameplay. Agentic frameworks further advance this direction by decoupling explicit game logic and controllable world representations from visual generation. However, existing approaches support only simple game settings and predefined motion patterns, generalize poorly to unseen environments and characters, and offer limited real-time interactivity. In this paper, we introduce Wizard, an agentic game world model that enables real-time, precise control over character motions while generalizing zero-shot across novel scenes, characters, and actions. To achieve this, we construct a diverse dataset spanning 805 scenes and 178 characters, incorporating skeleton and geometry information as structured proxies to condition video generation. Our agentic framework explicitly tracks and updates game states to support character-aware adaptive masking and world-anchored spatial conditioning, improving character identity preservation and spatial consistency. Experiments on WBench show that our method achieves higher gameplay fidelity with comparable video quality and spatiotemporal consistency, while using fewer parameters and less training data than state-of-the-art world models. These advantages are also retained in our two-step real-time variant. Overall, Wizard combines agentic control with structured proxy-based generation to enable player-driven game creation with real-time, generalizable, and mechanically consistent world simulation.

同方向论文 · cs.CV

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

Product Hunt · AI 分类67

Google 发布 Genie 4 智能体,可在虚拟 3D 世界中交互

Google 发布 Genie 4,这是一个由 Gemini 驱动的智能体,可在虚拟 3D 世界中与用户一起游玩、推理和学习。它不止执行基础指令,还能在交互环境中思考、理解并采取行动,用户可通过文本、语音或图像与它交流。

推荐理由:原文交代了 Genie 4 在虚拟 3D 环境中的交互方式,读者可据此判断它与只执行指令的智能体有何差别。