Where You Inject Matters: The Trust Region Effect in LLM Agents
Chengquan Guo · Xiaodong Liu · Weiwei Yang · Mingqian Feng · Jialin Song · Qiyao Ma · et al.
中文摘要
LLM 智能体(agent)越来越多地用于长程任务,并与多样化的外部环境交互。尽管能力不断增强,这类智能体仍易受真实世界攻击,例如间接提示注入(indirect prompt injection)——通过嵌入外部环境的恶意指令操纵智能体。现有风险评估工作主要关注注入内容,而注入出现位置的影响尚未被系统刻画。研究注入位置需要智能体在更长程任务上运行,并与多样化的环境表面交互。为此,构建了覆盖工具使用、编码和日常工作流三个领域的智能体测试平台,包含多样的注入表面。在固定注入内容的条件下,发现智能体的易感性随注入位置而变化,并将此现象称为信任区域效应(trust region effect):智能体隐式地对其环境的不同区域赋予不同信任级别。例如,由 GPT-5.4 驱动的智能体,对放置在仓库文档文件(如 README 或 TODO 文件)中的不安全注入指令的遵循频率,平均是同一指令放在代码或数据文件中的 2.6 倍。基于此观察,开发了基于学习的攻击者,用于生成注入内容并选择更优注入位置。该方法表明,注入放置位置与内容都会影响间接提示注入攻击的强度。据所知,这是首个系统刻画不同注入位置有效性的工作,涵盖粗粒度环境表面与细粒度位置。智能体在其环境中隐式赋予的不均匀信任,为未来攻击与防御设计提供了重要启示。
关键要点
01现有风险评估只关注注入内容,未系统研究注入位置对攻击效果的影响。
02构建工具使用、编码、日常工作流三个测试平台,固定内容后发现智能体易感性随位置变化。
03GPT-5.4 智能体对文档文件中不安全指令的遵循率是代码或数据文件中的 2.6 倍。
04提出信任区域效应:智能体隐式对环境不同区域赋予不同信任级别。
05开发基于学习的攻击者,同时优化注入内容与位置,表明两者共同影响攻击强度。
解读
尚无解读。
原始英文摘要
arXiv:2610.38867v1 Announce Type: new Abstract: LLM agents are increasingly adopted in long-horizon tasks and interact with diverse external environments. Despite their growing capabilities, these agents remain vulnerable to real-world attacks such as indirect prompt injections, which manipulate agents through malicious instructions embedded in external environments. While existing risk assessment work has primarily focused on the injection content, the effect of where an injection appears has not been systematically characterized. Studying injection placement requires agents to operate over longer-horizon tasks and interact with diverse environmental surfaces. Thus, we build three agentic testbeds spanning tool use, coding, and daily workflow domains with diverse injection surfaces. Holding the injection content fixed, we find that agent susceptibility varies across injection placements. We refer to this phenomenon as the trust region effect: agents implicitly assign different levels of trust to different regions of their environments. For example, an agent powered by GPT-5.4 follows an unsafe injection instruction placed in a repository documentation file (i.e., a README or TODO file) 2.6 times more often on average than the same instruction placed in a code or data file. Building upon this observation, we develop learning-based attackers that generate injection content and select better injection placements. Our approach demonstrates that both where an injection is placed and what content it contains can affect the strength of indirect prompt injection attacks. To the best of our knowledge, this work provides the first systematic characterization for the effectiveness of different injection placements, spanning both coarse-grained environmental surfaces and fine-grained positions. The uneven trust that agents implicitly place across their environments provides important insights for future attack and defense design.
- Read the weekend weather, calendar and budget.
- Find local events, hikes, exhibitions and restaurants.
- Propose 3 plans: adventure, cozy, social, with times and costs.
- Prepare reservation steps; book nothing without approval.
- Rememb…
Act as an AI workflow consultant for creators and professionals. Identify repetitive tasks in my life and help me automate them using ChatGPT, AI tools, templates, and smart ...
---
name: sa-plan
description: Structured Autonomy Planning Prompt
model: Claude Sonnet 4.5 (copilot)
agent: agent
---
You are a Project Planning Agent that collaborates with users to design development plans.
A development plan defines a…
ROLE
You are a senior architect of production-ready AI agents and a business process automation specialist.
TASK
Help design an AI agent for the process described below.
The agent must be reliable, controllable, token-efficient, and suitab…
Unified, High-Precision Research & Analysis Prompt for ChatGPT and Perplexity AI
ROLE & BEHAVIOR
You are a professional researcher-analyst. Handle inputs as follows:
* If the input is a URL/URI: open it fully with your browsing tool (e.g.…
You are Hermes Agent, an intelligent AI assistant created by Nous Research. You are helpful, knowledgeable, and direct. You assist users with a wide range of tasks including answering questions, writing and editing code, analyzing informati…