cs.SE提交于 2026-10-28已译

UnityBench-Mini3D: 评估 LLM 智能体在 Unity 中的 3D 游戏开发能力

UnityBench-Mini3D: Evaluating LLM Agents for 3D Game Development in Unity

Yijun Zhou · Minghao Shao · Ke Zhang · Renkai Yuan · Hongbo Yuan · Bin Hu · et al.

中文摘要

游戏开发对编码智能体的挑战不同于通用软件工程，其正确性取决于引擎 API、序列化场景状态与运行时行为的交互，而非仅看源代码。现有基准主要面向 Godot 等平台，且依赖 LLM 裁判或粗粒度任务结果。UnityBench-Mini3D 包含 77 个基于七个专家定义技能类别（而非教程或类型）的项目级 Unity 任务，分为三个难度层级。每个任务包含完整 Unity 项目与自然语言需求，使用 1,178 个保留的人工编写里程碑测试进行评估，涵盖场景结构、组件配置及运行时行为。这种引擎内评估提供确定性部分得分，定位失败点，并区分部分实现与完整解决方案，无需主观裁判。在 21 个模型和 9 个编码框架中，简单任务平均通过率为 80.9%，困难任务降至 35.2%，总体仅为 57.7%。该基准提供了可复现的诊断测试平台，区分不同难度下的系统表现。

关键要点

01现有基准多依赖 LLM 裁判或粗粒度结果，缺乏对引擎 API 与运行时交互的细粒度评估。
02构建包含 77 个任务、1,178 个人工测试的基准，通过引擎内确定性评估实现部分得分与失败定位。
03在 21 个模型测试中，简单任务通过率 80.9%，困难任务仅 35.2%，总体通过率 57.7%。

解读

尚无解读。

原始英文摘要

arXiv:2610.35980v1 Announce Type: cross Abstract: Game development stresses coding agents in ways general software engineering does not: correctness depends on how engine APIs, serialized scene state, and runtime behavior interact, not on source code alone. Existing benchmarks mainly target platforms such as Godot and heavily rely on LLM judges or coarse task-level outcomes. We present UnityBench-Mini3D, a benchmark of 77 project-level Unity tasks organized around seven expert-defined skill categories rather than tutorials or genres, with three difficulty tiers. Each task pairs a complete Unity project with natural-language requirements and is evaluated using 1,178 held-out, human-authored milestone tests covering scene structure, component configuration, and runtime behavior. This in-engine evaluation provides deterministic partial credit, localizes failures, and distinguishes partial implementations from complete solutions without relying on subjective judges. Across 21 models and 9 coding harnesses, the average pass rate reaches 80.9% on easy tasks but falls to 35.2% on hard tasks, with just 57.7% overall. UnityBench-Mini3D provides a reproducible, diagnostic testbed distinguishing systems across difficulty levels.