跳到正文
返回论文列表
cs.IR提交于 已译

通过子序列关联过滤与广度感知选择追溯并抑制LLM查询扩展中的幻觉词项

Tracing and Taming Hallucinated Terms via Subsequence-Association Filtering and Breadth-Aware Selection for LLM Query Expansion

Amit Kumar Jaiswal · Sagar Uprety · Prayag Tiwari

中文摘要

大语言模型(LLMs)已成为查询扩展(query expansion)词项的标准来源,但会产生幻觉。它们会注入并不存在的实体和跑题词项,带来排序噪声并可能损害强检索器的效果。现有方案,如语料库锚定、互验、偏好对齐,将幻觉词项当作无差别的干扰,通过一致性启发式过滤,既未给出某词项为何出现的机制性解释,也缺乏保留多少词项的原则性标准。本文提出子序列关联过滤的广度感知查询扩展(SAFE-QE),一种无需训练、与检索器无关的方法,基于幻觉的子序列关联(subsequence-association)解释,对每个候选扩展词项按其在语料库条件下的跨上下文复现性打分:在随机化查询上下文中重复出现但在伪相关集中缺乏支撑的词项被标记为主导幻觉关联的产物并予以降权。基于将语言生成刻画为一致性(无幻觉)与广度(覆盖)之间权衡的思路,将词项选择建模为伪相关集上的约束覆盖最大化,保留一组近似宽广但一致、且在重采样下稳定的词项。本文还提出幻觉扩展词项率(Hallucinated Expansion Term Rate),一种查询侧忠实度指标,统计相关文档均不支持的注入扩展词项数量。在TREC与BEIR基准集合上,SAFE-QE在nDCG@10上与生成式和语料库导向的扩展在统计上不可区分,同时每查询注入的未支撑词项更少:SAFE-QE为4.3,而Query2doc为27.8,HyDE为57.5;在强学习稀疏检索器上,所有扩展方法均出现退化,而忠实度提升并未伴随排序质量的可测量损失。

关键要点

  1. 01问题:LLM用于查询扩展时会产生幻觉词项,既有方法将幻觉视为无差别噪声,缺乏机制性解释与保留词项数的原则性标准
  2. 02方法:SAFE-QE无需训练且与检索器无关,基于子序列关联的幻觉解释,通过跨上下文复现性评分降权幻觉词项,并将词项选择建模为伪相关集上的约束覆盖最大化
  3. 03方法:提出幻觉扩展词项率(HETR),作为查询侧忠实度指标,统计相关文档不支持的注入扩展词项
  4. 04结果:在TREC与BEIR集合上,SAFE-QE的nDCG@10与生成式及语料库导向方法统计相当,未支撑词项由Query2doc的27.8、HyDE的57.5降至4.3/查询
  5. 05局限:在强学习稀疏检索器上,包括SAFE-QE在内的所有扩展方法均出现性能退化,忠实度提升未转化为排序收益

解读

尚无解读。

原始英文摘要

arXiv:2610.13810v1 Announce Type: cross Abstract: Large language models (LLMs) are now a standard source of query expansion terms but they hallucinate. They inject non existent entities and off-topic terms that add ranking noise and can hurt strong retrievers. Existing remedies such as corpus grounding, mutual verification and preference alignment treat hallucinated terms as an undifferentiated nuisance to be filtered by agreement heuristics, without a mechanistic account of why a given term appears or a principled criterion for how many terms to keep. We propose Subsequence-Association Filtered, breadth-aware query expansion (SAFE-QE), a training-free, retriever agnostic method that draws on the subsequence-association account of hallucination. We score each candidate expansion term by its corpus conditioned cross-context reproducibility, terms that reappear under randomised query contexts yet lack support in the pseudo-relevant set are flagged as products of dominant hallucinatory associations and down-weighted. Drawing on the characterisation of language generation as a trade-off between consistency (no hallucination) and breadth (coverage), we cast term selection as constrained coverage maximisation over the pseudo-relevant set, keeping a set that is approximately broad but consistent and stable across resampling. We also introduce the Hallucinated Expansion Term Rate, a query side faithfulness metric that counts injected expansion terms no relevant document supports. We evaluate our approach on benchmarked TREC and BEIR collections. Our SAFE-QE is statistically indistinguishable from generative and corpus-steered expansion on nDCG@10 while injecting fewer unsupported terms 4.3 per query against 27.8 for Query2doc and 57.5 for HyDE and on a strong learned sparse retriever, where every expansion method regresses. Faithfulness is therefore available at no measurable cost in ranking quality.

同方向论文 · cs.IR

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 2

数据aws.amazon.com·外部收录·

为生成式BI描述数据表和业务语义,提高大模型生成准确SQL的效率

明确告知大模型字段类型、字段属于维度还是度量、关联键信息、维度枚举值的业务含义以及枚举值是否可穷举;使用数据库中的真实值替代问题中的条件,并正确应用指标过滤条件。例如,将“上海地区”映射为 shanghai,将“男装”映射为 IN ('boy shirt', 'boy shoes', 'boy pants', 'boy coat'),且销售金额仅统计 status = 'finished' 的已完成订单。

数据infoq.cn·外部收录·

根据业务问题、候选表列及选表规则,智能决策最合适的数据表。

LLM 角色和任务描述 Freewheel 广告业务介绍 术语黑话映射表: 包括了常见的黑话和通过 RAG 方式召回的部分 数据目录与仓库概述 候选表和列:也就是上一步召回的表和列,包括每个表的描述和选表逻辑 通用选表规则:应该遵守的规则,例如什么时候应该选择事实表或者维度表,什么时候应该选择多张表 选表样例:我们收集了多样化的问题并标注了最优选择的表,存储到选表样例向量数据库中,通过 RAG 的方式找到和用户提问相似的例子。