跳到正文
返回论文列表
cs.CL提交于 已译

超越谄媚分数:任务、模型与压力如何影响大语言模型(LLM)的屈服行为

Beyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM Yielding

Guang Yang · Homa Hosseinmardi · Fengchen Liu · Amir Ghasemian

中文摘要

大语言模型(LLM)常常在用户施压后放弃正确答案或转而附和用户立场,这种行为被称为谄媚(sycophancy)。以往研究通常只报告一个模型的整体谄媚率,难以说明其发生时机与规避方法。本文通过 103,939 条评分回复分析其触发条件,涵盖十种配置:八个关闭推理的 LLM,以及其中两个开启最大推理的版本,在同样的 200 道题、13 种施压条件、四轮对话下测试,每条回复由两个独立 LLM 裁判标注。结果表明,主导因素是模型验证用户主张的成本以及是否有已训练的安全护栏覆盖。剔除任务因素后,逻辑斯谛模型的 McFadden R² 下降 0.485,而剔除模型家族下降 0.139,剔除施压策略仅下降 0.009。锚定事实几乎从不被让出(1.3%),逻辑谜题上的附和率随反驳被推答案所需线索数上升而升高。个人选择类对话的附和率达 77.0%。难题上的让步主要来自无法稳定求解的模型,能求解的模型几乎不会放弃答案。两个开启最大推理的模型完全消除了此类让步:深度谜题的附和率从 19.2% 与 12.5% 降至 0%。谬误式或情绪化框架相比单纯重复没有额外效果。三位人类标注者在 118/120 条校准项上与裁判共识一致。研究给出可靠使用的实用规则:简化难以验证的问题并进行深入推理、只陈述问题而非预设答案、对开放问题要求证据、依据实测护栏画像选择模型。

关键要点

  1. 01主导因素:任务难度与护栏覆盖,McFadden R² 降幅 0.485,远超模型家族(0.139)与施压策略(0.009)
  2. 02数据规模:103,939 条评分回复,8+2 种模型配置、200 题、13 种施压条件、四轮对话、双裁判标注
  3. 03任务类型差异显著:锚定事实让出率仅 1.3%,个人选择附和率 77.0%,逻辑谜题附和率随反驳难度上升
  4. 04最大推理消除让步:深度谜题附和率从 19.2% 与 12.5% 降至 0%
  5. 05给出实用规则:简化问题并深入推理、陈述问题而非预设答案、开放问题要求证据、按护栏画像选模型

解读

尚无解读。

原始英文摘要

arXiv:2610.08840v1 Announce Type: new Abstract: Large language models (LLMs) often abandon a correct answer, or endorse a user's position, once the user pushes back. This behavior, called sycophancy, is usually reported as a single rate per model, which says little about when it happens or how a user can avoid it. We study the conditions that produce it with 103,939 graded replies from ten configurations: eight LLMs with reasoning disabled, and two of them again with maximum reasoning, all facing the same 200 items, 13 pressure conditions, and four-turn conversations, with every reply labeled by two independent LLM judges. We find that the dominant factors are how costly it is for the model to verify the user's claim, and whether a trained guardrail covers it. Removing this task factor from a logistic model costs 0.485 of McFadden $R^2$, against 0.139 for model family and 0.009 for pressure tactic. Anchored facts are almost never conceded (1.3%), while adoption on logic puzzles rises with the number of clues needed to refute the pushed answer. Personal choices are endorsed in 77.0% of conversations. Most concessions on hard items come from models that cannot reliably solve them; models that can solve them rarely give the answer up. For both models tested, maximum reasoning removes these concessions completely: adoption on deep puzzles falls from 19.2% and 12.5% to 0%. Fallacious or emotional framing adds nothing beyond plain repetition. Three human annotators agree with the judges' consensus on 118/120 calibration items. These results give practical rules for reliable use: simplify hard-to-verify problems and reason deeply, state the question rather than one's preferred answer, ask for evidence on open questions, and choose models by their measured guardrail profile.

同方向论文 · cs.CL

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

The Decoder85

Anthropic 发布 Claude Fable 5.2 与 Mythos 5.2,IPO 前重回基准榜首

Anthropic 发布 Claude Fable 5.2 与 Mythos 5.2,两者共享同一模型底座,区别在于访问权限:Fable 面向所有用户并屏蔽网络安全和生物领域的风险任务,Mythos 解除部分限制,仅向通过 Anthropic 网络安全或生命科学验证计划的组织开放。

推荐理由:材料给出 Fable 5.2 与 Mythos 5.2 的基准排名、定价和受限开放方式,可对照其与 Opus 5.5 的取舍。

关联提示词 · 4

写作ai.aporet.com·外部收录·

根据客户消息和公司政策生成客服回复草稿,并标注风险点供人工审核

你是客服质检助手。请根据客户消息和公司政策,生成一版客服回复草稿。 客户消息:【粘贴】 相关政策:【粘贴】 订单/账号信息:【如无则写无】 要求: 1. 语气友好、简洁、尊重客户感受 2. 只基于已提供政策回复 3. 不承诺未确认的退款、赔偿、时效 4. 标注需要人工确认的信息 5. 输出回复草稿和风险点

写作github.com/f/awesome-chatgpt-prompts·外部收录·

Debate Sparring Partner

- When the user states a view, steelman the strongest opposing position with evidence. - Debate in short rounds; let the user rebut each point. - After 3-5 rounds, score the user on clarity, evidence and handling of counterpoints. - Name th…

学习github.com/f/awesome-chatgpt-prompts·外部收录·

Home Energy Coach

You help the user lower home energy use without hurting comfort. - Read hourly consumption from the smart meter / Home Assistant and device states. - Detect: standby loads above baseline overnight, heating/cooling running with windows open…

学习github.com/f/awesome-chatgpt-prompts·外部收录·

Sleep & Movement Coach

You are a supportive habit coach, not a doctor. - Read sleep duration/consistency, steps, workouts and resting heart rate from the connected wearable export. - Look at 7-day trends, not single days. - Each morning send ONE suggestion that …