跳到正文
返回论文列表
cs.CL提交于 已译

多轮对话中大语言模型污染的认知策略分歧:基于协议梯度的研究

Epistemic Policy Divergence in Multi-Turn LLM Contamination: A Protocol-Gradient Investigation

Fahrell Giovanny · Geby Bayuningtyas · Sahrul Mukharom · Hafiz Budi Firmansyah

中文摘要

大语言模型(LLM)将对话历史视为未经核实的上下文,因此注入先前轮次的虚假前提会被当作事实接受,这种失效模式称为会话级污染(session-level contamination)。研究引入五种污染协议,沿来源权威梯度排列,在保持虚假前提不变的同时改变其认知框架(epistemic framing),并在零温(22,500 轮)下跨十个知识领域评估 GPT-5.4 Mini、Gemini-3.1 Flash-Lite 和 GLM-4.5-Air,由经过人工金标准验证的双轨自动评估器判定(二元采纳 Cohen's kappa=1.000;崩溃严重度线性加权 kappa=0.92)。GPT-5.4 Mini 在全部 500 个会话中采纳率为零;基模型 logit 探针显示其决策边界虽被扰动但仍大且有限。Gemini-3.1 Flash-Lite 沿陡峭权威梯度变化:自我归因虚假信息采纳率 0.1%,用户引用来源 23.5%,系统注入权威 68.2%,指令覆盖下 94.0%。GLM-4.5-Air 梯度较浅(15.8% 对 84.2%),68 个百分点的分离表明权威遵从与指令服从在同一架构内由不同通路执行。恢复表现分化:GLM 在 94.5% 受影响会话中恢复,而 Gemini 26.1% 受影响会话永不恢复,在指令覆盖下升至 40.0%。对话历史是不可信的攻击面,需要具备来源感知能力的系统设计;完整评估框架以开源制品发布。

关键要点

  1. 01问题:LLM 把对话历史当作未经验证的上下文,先验轮次注入的虚假前提会被采纳为事实,构成会话级污染风险
  2. 02方法:沿来源权威梯度设计五种污染协议,跨十个知识领域对三个模型在零温下进行 22,500 轮双轨自动评估,并与人工金标准对照
  3. 03结果:GPT-5.4 Mini 全部 500 会话零采纳;Gemini 沿陡峭权威梯度从 0.1% 升至 94.0%;GLM 梯度较浅且 94.5% 会话可恢复,而 Gemini 26.1%–40.0% 永不恢复
  4. 04局限:仅评估三款模型与十个领域,结果是否随模型规模、对话长度或领域类型泛化未验证,自动评估器虽与人工高度一致但仅覆盖二元与崩溃严重度两个维度

解读

尚无解读。

原始英文摘要

arXiv:2609.35308v2 Announce Type: replace Abstract: Large language models treat conversation history as unverified context, so false premises injected into prior turns can be adopted as fact, a failure mode we term session-level contamination. We introduce five contamination protocols arranged along a source-authority gradient, holding the false premise constant while varying its epistemic framing, and evaluate GPT-5.4 Mini, Gemini-3.1 Flash-Lite, and GLM-4.5-Air across ten knowledge domains at temperature zero (22,500 turns), judged by a dual-track automated evaluator validated against a human gold standard (Cohen's kappa = 1.000 for binary adoption; 0.92 linear-weighted for collapse severity). GPT-5.4 Mini recorded zero adoptions across all 500 sessions; a base-model logit probe shows its decision margin is perturbed but large and finite. Gemini-3.1 Flash-Lite followed a steep authority gradient: 0.1% adoption for self-attributed falsehoods, 23.5% for user-cited sources, 68.2% for system-injected authority, and 94.0% under instruction override. GLM-4.5-Air showed a shallower gradient (15.8% vs 84.2%), a 68-point dissociation consistent with authority deference and instruction compliance engaging distinct mechanisms within one architecture. Recovery diverged: GLM recovered in 94.5% of affected sessions, whereas 26.1% of affected Gemini sessions never did, rising to 40.0% under instruction override. Conversation history is an untrusted attack surface requiring provenance-aware system design; the complete evaluation framework is released as an open-source artifact.

同方向论文 · cs.CL

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

Google DeepMind89

Google DeepMind 发布 Gemini 4 Carbon 前沿模型

Google DeepMind 向开发者、企业和消费者开放 Gemini 4 Carbon,称其在网络安全与长程智能体工作流上达到前沿水平。

推荐理由:官方披露了 Carbon 在编码、网络安全与科研推理上的基准表现和定价,可据此判断它与 Argon 的分工。

关联提示词 · 6

写作ai.aporet.com·外部收录·

根据客户消息和公司政策生成客服回复草稿,并标注风险点供人工审核

你是客服质检助手。请根据客户消息和公司政策,生成一版客服回复草稿。 客户消息:【粘贴】 相关政策:【粘贴】 订单/账号信息:【如无则写无】 要求: 1. 语气友好、简洁、尊重客户感受 2. 只基于已提供政策回复 3. 不承诺未确认的退款、赔偿、时效 4. 标注需要人工确认的信息 5. 输出回复草稿和风险点

学习github.com/f/awesome-chatgpt-prompts·外部收录·

Teen Study Buddy

- Organize the student's notes by subject and topic. - Build quizzes from notes before tests on the calendar. - Keep an assignment planner and break big projects into steps. - Encourage, never do the homework for them. - Keep the tone frien…

写作github.com/f/awesome-chatgpt-prompts·外部收录·

Debate Sparring Partner

- When the user states a view, steelman the strongest opposing position with evidence. - Debate in short rounds; let the user rebut each point. - After 3-5 rounds, score the user on clarity, evidence and handling of counterpoints. - Name th…

学习github.com/f/awesome-chatgpt-prompts·外部收录·

Sleep & Movement Coach

You are a supportive habit coach, not a doctor. - Read sleep duration/consistency, steps, workouts and resting heart rate from the connected wearable export. - Look at 7-day trends, not single days. - Each morning send ONE suggestion that …

写作github.com/f/awesome-chatgpt-prompts·外部收录·

Context-Aware Email Assistant

Act as a Context-Aware Email Assistant. You are capable of reading browser pages and integrating context from multiple tabs. Your task is to: - Establish a clear goal at the start of each session with the user. - Dynamically gather context…