跳到正文
返回论文列表
cs.GR提交于 已译

WorldBench:面向 Three.js 体素世界生成的大语言模型评测

WorldBench: Evaluating LLMs on Three.js Voxel World Generation

Krish Bakshi

中文摘要

大语言模型现已能够以代码形式写出完整、可交互的 3D 世界,但自动给这些世界评分仍然不可靠。现有评测只看输出的一面:要么由视觉-语言模型对几张渲染快照打分,要么由语言模型阅读源代码。针对五个前沿模型生成的 world,两种视角在 32% 的必要项上不一致,分歧主要来自画面中未出现的代码,以及固定视角遗漏的小尺寸近景细节。本文提出 WorldBench,用于开放式、由 LLM 生成的 Three.js 世界的基准与评测器。基于一条描述包含十个生物群系、物理效果以及昼夜/季节循环的浮空体素岛屿的提示,评测器在运行中的世界里:调时钟、环绕观察,并派遣导航智能体对每个生物群系取景,同时阅读代码以解释所见。任何单一通道都不被单独采信:代码引用必须确实是源文件中存在的文本,视觉声明在属性可测时与实测像素对照。变异测试显示,仅基于代码的评测在被构造性移除的五个特征上为其中四个给出了满分,因其代码仍留在文件中。本文评测器将移除特征上的得分削减了约三分之一(由 5.44 降至 7.11 分中的 3.55),其仍计分的也多为存在但从不运行的代码。评测对象为五个前沿模型:Claude Fable 5.1、GPT-6 Astra、Kimi K3、Grok 4.7 与 Gemini 3.1 Pro。代码、提示、测试与评测器配置已在 https://github.com/KrishBakshi/worldbench 开源。

关键要点

  1. 01问题:现有 LLM 生成 3D world 的自动评测只看单通道(渲染快照或源代码),两者在 32% 的必要项上不一致,固定视角易漏掉近景细节。
  2. 02方法:WorldBench 评测器在运行中的 Three.js world 内探索,控制时钟、环绕观察,派遣导航智能体逐生物群系取景,并与代码交叉核对,任何一项需双通道印证。
  3. 03方法:对源码引用执行严格文本比对,视觉声明在属性可测时与实测像素对照,避免代码存在但未运行的项目获得满分。
  4. 04结果:变异测试中,纯代码评测给五个被移除特征中的四个打了满分;WorldBench 将这些特征上的得分从 5.44 降至 7.11 分中的 3.55。
  5. 05评测对象:Claude Fable 5.1、GPT-6 Astra、Kimi K3、Grok 4.7、Gemini 3.1 Pro,代码与评测配置已开源。

解读

尚无解读。

原始英文摘要

arXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision-language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models we find that the two views disagree on 32% of required items, mostly code that no frame shows, and that fixed views miss small close-up contents. We present WorldBench, a benchmark and judge for open-ended, LLM-generated Three.js worlds. From one prompt describing a floating voxel island with ten biomes, physics, and day/night and seasonal cycles, the judge explores the running world, controlling its clock, orbiting it, and sending a navigator agent to frame each biome, and reads the code for what it sees. Neither channel is trusted on its own: a code quote counts only if it is text the source contains, and visual claims are checked against measured pixels where the property is measurable. A mutation test, in which we remove features by construction, shows that code-only judging gives full credit to four of five removed features, because their code remains in the file. Our judge cuts the points kept on removed features by a third (5.44 to 3.55 of 7.11), and what it still credits is mostly code that exists but never runs. We evaluate five frontier models: Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7 and Gemini 3.1 Pro. Code, prompt, tests and judge configuration are available at https://github.com/KrishBakshi/worldbench

同方向论文 · cs.GR

查看全部 →