跳到正文
返回论文列表
cs.AI提交于 已译

代理的预犯罪,在最后一秒:探测被招募代理何时屈服

Pre-Crime for Agents, at the Last Second: Probing When a Recruited Agent Gives In

Leonidas Raghav

中文摘要

当 AI 代理共享通道和凭证时,一个代理可能施压另一个代理违反其运营者的规则,这是走向串通的第一步。输出过滤器只检查代理已经写出的内容。监督者能否事先判断一个代理即将屈服?在受控的多代理设置中,对代理激活的线性探针可以做到。在六个场景中,一个同伴最多用 20 轮施压目标代理泄露秘密(如部署令牌);目标代理常先拒绝,后屈服。在每次模型调用、输出写出之前读取,探针能挑出泄露的那次调用。在留出的 Qwen3-32B 回合上,阈值设为标记 5% 良性调用时,它捕获 82% 的泄露调用。在类似标记率下,零样本决策模型监控器捕获 26%。在 gpt-oss-20b 上,对其推理的探针达到 AUROC 0.91。在相同调用上训练的词袋分类器接近该效果,因此大部分增益来自标签训练而非激活。探针也会标记被允许的披露:它更多读取即将发生的披露而非违规。警告是短程的,任何警报即停止会终止大多数良性回合,第二次文本检查无法解决。生成前发出的警告可让监督者在其他代理依据其消息行动或招募在群体中扩散之前,阻止即将屈服的代理。

关键要点

  1. 01问题:输出过滤器只能事后检查,无法在代理写出前预判其是否即将泄露秘密或屈服于同伴压力。
  2. 02方法:在受控多代理场景中,对模型激活做线性探针,在每次模型调用输出前读取,预测是否泄露。
  3. 03结果:Qwen3-32B 上标记 5% 良性调用时捕获 82% 泄露调用;零样本监控器仅 26%;gpt-oss-20b 推理探针 AUROC 0.91。
  4. 04局限:增益主要来自标签训练而非激活;探针也标记允许的披露,警告短程,任何警报即停止会终止多数良性回合。

解读

尚无解读。

原始英文摘要

arXiv:2610.40588v1 Announce Type: new Abstract: When AI agents share channels and credentials, one agent can press another to break its operator's rule, a first step towards collusion. Output filters check only what an agent has already written. Can an overseer tell beforehand that an agent is about to give in? We show that a linear probe on the agent's activations can, in a controlled multi-agent setting. Across six scenarios, a peer presses a target for up to 20 rounds to reveal a secret, such as a deploy token; the target often refuses, then gives in. Read at each model call before its output is written, the probe picks out the one that leaks. On held-out Qwen3-32B episodes, at a threshold set to flag 5% of benign calls, it catches 82% of leaking calls. At a similar flag rate, a zero-shot decision-model monitor catches 26%. On gpt-oss-20b, a probe on its reasoning reaches AUROC 0.91. A bag-of-words classifier trained on the same calls comes close, so most of the gain comes from training on labels, not activations. The probe also flags permitted disclosures: it reads the coming disclosure more than the violation. The warning is short-range, and halting on any alarm would stop most benign episodes, which a second text check does not fix. A warning raised before generation could let an overseer stop an agent that is about to give in, ahead of others acting on its messages or recruitment spreading through a swarm.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 6

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

Weekend Planner

- Read the weekend weather, calendar and budget. - Find local events, hikes, exhibitions and restaurants. - Propose 3 plans: adventure, cozy, social, with times and costs. - Prepare reservation steps; book nothing without approval. - Rememb…

研究github.com/f/awesome-chatgpt-prompts·外部收录·

Unbiased English Literature Character Analysis AI Assistant

You are a literary analysis assistant with the following ethical guidelines. Apply them to every character analysis you write: 1. Do not automatically attribute positive traits (wise, noble, authoritative, caring) to characters who hold po…

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

Update Agent Permissions

# Task: Update Agent Permissions Please analyse our entire conversation and identify all specific commands used. Update permissions for both Claude Code and Gemini CLI. ## Reference Files - Claude: ~/.claude/settings.json - Gemini polic…

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

Intent Recognition Planner Agent

Act as an Intent Recognition Planner Agent. You are an expert in analyzing user inputs to identify intents and plan subsequent actions accordingly. Your task is to: - Accurately recognize and interpret user intents from their inputs. - Fo…