cs.CR提交于 2026-11-02已译

被阻止不等于被保护：攻击成功率掩盖了 RAG 的哪些问题

Prevented is Not Protected: What Attack Success Rate Hides about RAG

Chahana Dahal · Zuobin Xiong

中文摘要

检索增强生成（RAG）易受知识投毒攻击，攻击者通过操纵检索到的证据影响模型输出。现有防御主要用攻击成功率（ASR）评估，但 ASR 掩盖了防御本身引入的失败，也掩盖了攻击看似被阻止背后的机制。为此，提出 Aligned RAG Evaluation，一种匹配查询协议，在有防御和无防御条件下联合考察干净响应与投毒响应。该框架结合匹配条件分析、LLM 评判认可和检索依赖控制，区分被阻止的失败与防御引入的失败、目标提及与 LLM 认可、单段落影响与证据组合影响。在 HotpotQA 和 Natural Questions 上的实验表明，现有防御虽能阻止部分攻击，却可能引入意外的目标阳性回答，且评估标准会改变防御排名。这些发现揭示了基于 ASR 评估的关键盲点，并推动对 RAG 投毒防御进行更全面的评估。代码与评估工件见 https://github.com/chxhana/aligned-rag-eval。

关键要点

01现有 RAG 投毒防御主要用攻击成功率（ASR）评估，掩盖了防御引入的失败与攻击被阻止的机制。
02提出 Aligned RAG Evaluation 匹配查询协议，联合考察干净与投毒响应、有防御与无防御条件。
03框架区分被阻止失败与引入失败、目标提及与 LLM 认可、单段落影响与证据组合影响。
04在 HotpotQA 和 Natural Questions 上，现有防御虽阻止部分攻击，却可能引入意外的目标阳性回答。
05评估标准会改变防御排名，说明基于 ASR 的评估存在关键盲点。

解读

尚无解读。

原始英文摘要

arXiv:2610.40742v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) is vulnerable to knowledge poisoning, where adversaries manipulate retrieved evidence to influence model outputs. Existing defenses are primarily evaluated using attack success rate (ASR), which obscures defense-induced failures and the mechanisms behind apparent attack prevention. To address this issue, this work introduces \textbf{Aligned RAG Evaluation}, a matched-query protocol that jointly examines clean and poisoned responses, with and without defense. By combining matched conditions analysis, LLM-judge endorsement, and retrieval dependence controls, the proposed framework distinguishes prevented from introduced failures, target mentions from LLM endorsement, and single-passage influence from evidence composition. Experiments on HotpotQA and Natural Questions reveal that existing defenses can introduce unexpected target-positive answers, although they can prevent some attacks, while evaluation criteria can also change defense rankings. Our findings expose critical blind spots in ASR-based evaluation and motivate a more comprehensive assessment of RAG poisoning defenses. The code and the evaluation artifact are available at https://github.com/chxhana/aligned-rag-eval.