SecPO：面向提示注入安全的原则性对抗训练
SecPO: Principled Adversarial Training for Prompt Injection Security
Sizhe Chen · Yibo Peng · Jaewon Chang · Chawin Sitawarin · David Wagner
中文摘要
提示注入（prompt injection）被公认为 AI 智能体的首要威胁，来自不可信数据（邮件、文档、网站）的指令可操纵智能体运行。现有防御最强的 LLM Meta-SecAlign 用 DPO 损失训练，但在自适应提示注入下攻击成功率（ASR）仍接近 100%。原因之一是 DPO 将模型锚定在不安全的无防御参考模型上。研究表明，理想化参考模型——一个注入不变且保持效用的 oracle——其实可以被近似，因为其训练时的输出分布可被有趣地获取。这一观察催生了理论上更有原则的 SecPO 损失。Meta-SecAlign 的另一缺陷是仅在弱模拟攻击上训练。我们实现高效对抗训练（AT）流水线，合成强注入输入（best@240）并用于训练。相比 Meta-SecAlign，AT 在三种 SoTA 自适应攻击下将 ASR 降低 1.5-14.5 倍；相比 DPO 损失，SecPO 损失进一步降低 2.4-4.6 倍。在智能体工作流中，SecPO + AT 的 Qwen3.6-27B 在 AttackerMovesSecond 攻击下 AgentDojo ASR@800 为 8.8%，远低于 Meta-SecAlign 的 78.8%。此外，防御模型在覆盖智能体工具调用、指令遵循、科学和数学的九个基准上良性效用无明显下降。代码、训练数据（含 AT 优化注入）和模型已发布。
关键要点
- 01DPO 将模型锚定在不安全的无防御参考模型上，导致自适应提示注入下 ASR 接近 100%
- 02理想化注入不变 oracle 的输出分布在训练中可被获取，据此提出 SecPO 损失
- 03AT 流水线合成强注入输入（best@240），在三种 SoTA 自适应攻击下 ASR 降低 1.5-14.5 倍
- 04SecPO 损失相比 DPO 损失进一步将 ASR 降低 2.4-4.6 倍
- 05AgentDojo ASR@800 为 8.8% 对 78.8%，且九个基准良性效用无明显下降
解读
尚无解读。
原始英文摘要
arXiv:2610.41998v1 Announce Type: cross Abstract: Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation. The existing best-defended LLM, Meta-SecAlign, is trained with the DPO loss but still suffers from attack success rates (ASRs) approaching 100% under adaptive prompt injections. One reason is that DPO anchors the model around the undefended reference model, which is insecure. We show an idealized reference model---an injection-invariant, utility-preserving oracle---can nevertheless be approximated, as its output distribution in training is interestingly accessible. This new observation motivates a theoretically principled SecPO loss. Another Meta-SecAlign shortcoming is that it trains on weak simulated attacks. We implement an efficient adversarial training (AT) pipeline that synthesizes strong injected inputs (best@240), and train the model on them. Our AT reduces ASRs by 1.5-14.5$\times$ across three SoTA adaptive attacks, compared with Meta-SecAlign; the SecPO loss reduces them by a further 2.4-4.6$\times$ compared with using DPO loss. In agentic workflows, our SecPO + AT Qwen3.6-27B is much more secure than Meta-SecAlign: 8.8% vs. 78.8% AgentDojo ASR@800 under the AttackerMovesSecond attack. Moreover, our defended model shows no noticeable reduction in benign utility in nine benchmarks covering agentic tool calling, instruction following, science, and math. We release our $\href{//github.com/Sizhe-Chen/SecPO}{\text{code}}$, $\href{//huggingface.co/datasets/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO-trainset}{\text{training data}}$ (with optimized injections from AT), and $\href{//huggingface.co/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO}{\text{model}}$.