跳到正文
返回论文列表
cs.CL提交于 已译

Gamer:面向长时程实时游戏智能体的通用视觉-语言-动作基础模型

Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent

Run Luo · Kuan Zhang · Qi Deng · Yukun Chen · Zhihao Yang · Yufei Zhang · et al.

中文摘要

Gamer 是面向复杂交互环境长时程任务的通用智能体框架,端到端架构整合感知、记忆管理与多步推理,共享训练与部署流程同时支持传统 2D 图形界面和复杂 3D 模拟世界。首先在由数万小时设备级键盘鼠标交互构建的数据集上继续预训练,轨迹覆盖模拟游戏与具身环境,使模型学习物理与虚拟域的环境动态。稀疏思考指令微调大幅减少冗余计算,同时保留复杂决策所需推理。策略采用稳定的多轮强化学习方法训练长时程交互,应对有限上下文窗口、累积 rollout 误差和稀疏奖励。实时交互方面,云边推理引擎并发服务多个模拟游戏环境,支持无界上下文执行并保持低在线推理延迟。在多个高难度 3D 开放世界游戏中,Gamer 展现出与熟练玩家相当的泛化能力和运行效率,在一系列传统 2D 游戏基准上也取得有竞争力的结果。这表明即使重复的键盘鼠标轨迹,也能通过扩展数据和学习算法带来显著收益,帮助 Gamer 在兼具物理与数字属性的环境中更稳定地学习复杂技能,为跨数字与物理世界交互的通用智能体提供路径。

关键要点

  1. 01问题:长时程复杂交互环境缺乏统一感知、记忆与推理的通用智能体框架。
  2. 02方法:数万小时键鼠轨迹继续预训练,结合稀疏思考指令微调与多轮强化学习。
  3. 03结果:3D 开放世界游戏泛化与效率接近熟练玩家,2D 游戏基准具竞争力。
  4. 04工程:云边推理引擎并发服务多环境,支持无界上下文并保持低延迟。
  5. 05局限:摘要未明确说明具体失败场景或性能上限。

解读

尚无解读。

原始英文摘要

arXiv:2611.18397v1 Announce Type: cross Abstract: We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds.

同方向论文 · cs.CL

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

Product Hunt · AI 分类67

Google 发布 Genie 4 智能体,可在虚拟 3D 世界中交互

Google 发布 Genie 4,这是一个由 Gemini 驱动的智能体,可在虚拟 3D 世界中与用户一起游玩、推理和学习。它不止执行基础指令,还能在交互环境中思考、理解并采取行动,用户可通过文本、语音或图像与它交流。

推荐理由:原文交代了 Genie 4 在虚拟 3D 环境中的交互方式,读者可据此判断它与只执行指令的智能体有何差别。