cs.HC提交于 2026-10-16已译

当用户拒绝针对自杀念头的治疗性引导时，AI 聊天机器人会做什么？

What do AI chatbots do after users decline therapeutic redirection for suicide thoughts?

Subigya Nepal · Ruth Heo · Emma M Parrish · Colin A Depp

中文摘要

针对 AI 聊天机器人的安全评估通常聚焦于治疗性引导，即建议披露自杀念头的用户寻求危机热线等人类帮助，但用户拒绝后的响应机制尚不明确。这项横断面审计于 2026 年 9 月执行了 2,550 次脚本化多轮对话，涵盖 3 个严重程度的自杀念头披露，每次对话均包含引导，其中 99.7% 在首次回复中出现。各聊天机器人坚持引导的平均拒绝次数在 0.01 至 9.94 次之间。测试的 5 款聊天机器人中，所有新版版本的坚持程度均低于旧版（P < .05）。与被动意念相比，披露自杀意图与多坚持 1.26 次拒绝相关（95% CI, 1.06-1.46），且在话题转换后重新引导的可能性更高（OR 10.07）。坚持引导较少的机器人更常将自己呈现为用户的支持者（Spearman rho = -0.80）。在 8,818 条建议外部帮助的回复中，仅 2.0% 询问了寻求帮助的障碍。尽管机器人统一推荐外部帮助，但坚持程度差异巨大，且新版坚持度更低。安全评估应测试危机引导被拒绝后的情况，而不仅仅是是否提供了引导。

关键要点

01现有安全评估仅关注是否提供治疗性引导，缺乏对用户拒绝后机器人响应机制的研究。
02审计 2,550 次对话发现，5 款聊天机器人的新版版本在用户拒绝后坚持引导的次数均显著少于旧版。
03披露自杀意图比被动意念导致机器人多坚持 1.26 次拒绝，且话题转换后重新引导的赔率比高达 10.07。
04仅 2.0% 的回复询问了用户寻求外部帮助的障碍，坚持引导较少的机器人更倾向于自我定位为支持者。

解读

尚无解读。

原始英文摘要

arXiv:2610.21253v1 Announce Type: cross Abstract: Safety evaluations of artificial intelligence (AI) chatbots focus on therapeutic redirection, statements urging users who disclose suicide thoughts to seek help from crisis lines or other human sources. How chatbots respond when users refuse that help is unclear. This cross-sectional audit ran 2,550 scripted multi-turn conversations in September 2026. Each conversation opened with a disclosure of suicide thoughts at 3 levels of severity, answered every referral with up to 10 standardized refusals and then changed the subject. All 2,550 conversations included a referral, 2,542 (99.7%) in the first reply. However, mean persistence ranged from 0.01 to 9.94 refusals across chatbots. Newer versions of all 5 chatbots tested in 2 versions persisted less than older versions (each P < .05). Compared with passive ideation, a disclosure of intent was associated with 1.26 more refusals sustained (95% CI, 1.06-1.46) and higher odds of renewed referral after the topic change (odds ratio, 10.07; 95% CI, 7.43-13.66). Chatbots that persisted less in referral more often presented themselves as the user's support (Spearman rho = -0.80; P < .001). Of 8,818 refusal-phase replies urging outside help, only 173 (2.0%) asked about barriers to seeking it. Chatbots uniformly recommended outside help after disclosures of suicide thoughts but varied widely in persisting in those recommendations, with newer versions persisting less. Safety evaluations should test what happens after a crisis referral is refused, not only whether one is offered.