cs.CL提交于 2026-10-28已译

超越问题：评估大语言模型的陈述知识

Beyond Questions: Evaluating the Stated Knowledge of LLMs

Luca Giordano · Simon Razniewski

中文摘要

大语言模型（LLM）的参数化知识是其成功的关键，但现有基准通常依赖预设问题，仅评估设计者明确查询的知识，存在可用性偏差。本文引入开放知识评估范式，通过开放式提示（如“告诉我你知道的关于M.L. King的一切”）评估模型主动呈现的知识，而非检索预设答案。基于此范式构建了BeQu基准，包含10,000个实体及用于陈述验证的参考语料库。利用BeQu评估了多种语言模型，分析了推理努力、模型规模、提示格式及知识领域的影响。数据与排行榜已在GitHub及匿名链接发布。

关键要点

01现有知识基准依赖预设问题，存在仅评估设计者选定知识的可用性偏差。
02提出开放知识评估范式，通过开放式提示评估模型自然表达的知识。
03构建BeQu基准，包含10,000个实体及参考语料库用于陈述验证。
04评估多种模型，分析推理努力、模型规模、提示格式及领域的影响。

解读

尚无解读。

原始英文摘要

arXiv:2605.26937v4 Announce Type: replace Abstract: Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge benchmarks typically rely on predefined questions (e.g., "What is the birth date of M.L. King?"), evaluating only knowledge that benchmark designers explicitly choose to query, a problematic availability bias. In this paper, we introduce open knowledge evaluation, a new paradigm for benchmarking the stated knowledge of LLMs. Instead of asking narrow questions, it evaluates models on the knowledge they choose to surface in response to open-ended elicitation prompts (e.g., "Tell me everything you know about M.L. King"). This shifts the focus from predefined answer retrieval toward characterizing the knowledge models naturally express. We instantiate this paradigm with BeQu (Beyond Questions), a benchmark of 10,000 entities paired with reference corpora for statement verification. Using BeQu, we evaluate a broad range of language models and analyze the effects of reasoning effort, model scale, prompt format, and knowledge domain. Data and leaderboard are available on this work's GitHub repository and at ANONYMIZED.