cs.AI提交于 2026-10-31已译

本体能捕获什么？精选、LLM 生成与数据派生本体作为知识图谱上 LLM 智能体检查器的比较

What Does an Ontology Catch? Curated, LLM-Generated and Data-Derived Ontologies as Checkers of LLM Agents over Knowledge Graphs

Michal Orzechowski · Rund Tawfiq · Tomasz Kosciolek

中文摘要

面向 Web 知识图谱问答的 LLM 智能体，越来越多地使用由图谱本体（ontology）派生的检查器，前提是形式化 schema 能充当可靠裁判。但检查器本身也可能出错：既可能漏检错误，也可能误报正确答案并促使智能体去“修正”它们。精选本体作为裁判是否值得其成本，以及它与 LLM 生成的本体或从数据中挖掘的 schema 相比如何，尚不清楚。该工作把检查器本身作为研究对象。在精选、LLM 生成和数据派生三类本体上，分别以提示上下文、检查器或两者兼用的方式，在公开图谱和近 10^9 三元组的私有图谱上，对开放权重与专有智能体进行评测，统计本体派生检查捕获了什么、误报了什么。研究展示各来源如何在捕获错误与误报之间权衡、按原样使用的精选 schema 如何与自身数据矛盾、结构检查会漏掉哪些错误，以及合并来源或向智能体提供检查器的具体反馈是否有收益。在 UniProt 上，精选本体比数据派生本体或由掩码标识符的数据摘要经 LLM 派生的本体标记出更多正确答案；而在真实智能体答案上，其标记的精确率仅略高于随机标记。

关键要点

01将本体派生检查器本身作为研究对象，系统评估精选、LLM 生成与数据派生三类本体在知识图谱问答中的表现。
02在公开图谱与近 10^9 三元组私有图谱上，对比开放权重与专有 LLM 智能体，统计漏检与误报。
03精选本体在 UniProt 上标记出更多正确答案，但在真实答案上精确率仅略高于随机。
04揭示按原样使用的精选 schema 可能与自身数据矛盾，且结构检查会漏掉特定类型错误。
05考察合并本体来源或向智能体提供检查器具体反馈是否能带来收益。

解读

尚无解读。

原始英文摘要

arXiv:2610.37809v1 Announce Type: new Abstract: LLM agents that answer questions over Web knowledge graphs are increasingly wrapped in checks derived from the graph's ontology, on the assumption that a formal schema makes a reliable judge. Yet a checker can be wrong as well: it can miss errors, and it can flag correct answers and push the agent into "fixing" them. Whether a curated ontology is worth its cost as such a judge, and how it compares with an ontology an LLM writes or a schema mined from the data, is unknown. We make the checker itself the object of study. Across curated, LLM-generated and data-derived ontologies, used as prompt context, as a checker, or both, and across open-weight and proprietary agents on public graphs and a private graph of nearly $10^{9}$ triples, we account for what ontology-derived checks catch and what they wrongly flag. We show how each source trades caught errors against false alarms, how a curated schema used as published can contradict its own data, which errors structural checks miss, and whether combining sources or giving the agent the checker's specific feedback pays off. On UniProt, the curated ontology flags more correct answers than either an ontology derived from the data or one an LLM derives from a summary of the data with masked identifiers. On real agent answers, its flags are only marginally more precise than random flags.