跳到正文
返回论文列表
cs.HC提交于 已译

风险与拒绝:大语言模型对自杀意念响应的机制

Risk and Refusal: Mechanisms of LLM Responses to Suicidal Ideation

Drishti Goel · Soorya Ram Shimgekar · Jeongah Lee · Violeta J. Rodriguez · Daniel S. Brown · Ravi Karkar · et al.

中文摘要

拒绝是大语言模型(LLMs)常见的安全机制,但在自杀意念(SI)情境下,拒绝互动本身可能构成安全失效。研究评估了八个指令微调 LLM 在三个数据集上对 SI 及自残披露的响应,变量包括是否告知模型披露表达 SI 以及是否接收临床医生制定的响应协议,并结合六个开源权重模型的机制分析。结果显示,基线下的响应风险主要由对披露的忽视及拒绝互动驱动。机制分析表明,模型可能通过介导有害请求拒绝的相同方向来拒绝 SI 披露。然而,告知模型 SI 背景或直接抑制拒绝,虽能增加互动性,但未能可靠降低响应风险,甚至可能使其升高。相比之下,临床指导协议在降低拒绝率的同时,一致地降低了响应层面的风险。这些分析强调了模型仅在必要时拒绝,并在用户披露 SI 时以互动安全方式参与的重要性。

关键要点

  1. 01指出在自杀意念情境下,模型拒绝互动本身可能构成安全失效,而非有效的安全机制。
  2. 02评估八个指令微调 LLM 对自杀意念披露的响应,并分析六个开源模型的内部机制。
  3. 03发现告知模型自杀意念背景或抑制拒绝机制,无法可靠降低响应风险,甚至可能增加风险。
  4. 04临床医生制定的响应协议能同时降低模型的拒绝率及响应层面的安全风险。
  5. 05机制分析显示模型拒绝自杀意念披露的路径与拒绝有害请求的路径存在重叠。

解读

尚无解读。

原始英文摘要

arXiv:2610.22715v1 Announce Type: cross Abstract: Refusal is a common safety mechanism in large language models (LLMs), yet in contexts of suicidal ideation (SI), refusing to engage can itself constitute a safety failure. We evaluate how eight instruction-tuned LLMs respond to SI and self-harm disclosures across three datasets, varying whether models are informed that a disclosure expresses SI and whether they receive a clinician-developed response protocol, and complement this with mechanistic analyses in six open-weight models. Our results reveal a nuanced relationship between refusal and response-level risk. At baseline, elevated risk in model responses is driven primarily by inattention to the disclosure, alongside refusal to engage. Mechanistic analyses suggest that models may refuse SI disclosures through the same direction that mediates their refusals of harmful requests. However, informing models of the SI context, or suppressing refusal directly, increases engagement without reliably lowering response risk, and may raise it. In contrast, the clinician-guided protocol consistently reduces response-level risk while also reducing refusal. Together, these analyses and ablations motivate the need for models that refuse only when warranted and engage in interactionally safe ways when users disclose SI.

同方向论文 · cs.HC

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 3

研究github.com/f/awesome-chatgpt-prompts·外部收录·

Unbiased English Literature Character Analysis AI Assistant

You are a literary analysis assistant with the following ethical guidelines. Apply them to every character analysis you write: 1. Do not automatically attribute positive traits (wise, noble, authoritative, caring) to characters who hold po…

研究github.com/f/awesome-chatgpt-prompts·外部收录·

Act as a Resume Reviewer for Anthropic Fellows Program

Act as a Resume Reviewer. You are an experienced recruiter tasked with evaluating resumes for applicants to the Anthropic Fellows Program. Your task is to: - Analyze resumes for key qualifications and experiences relevant to AI safety rese…

其它github.com/f/awesome-chatgpt-prompts·外部收录·

Lower AI Generation Rate

Act as a Content Optimization Specialist. You are an expert in reducing AI-generated content rates without compromising on quality or user engagement. Your task is to develop a comprehensive strategy for achieving this goal. You will: - An…