模型知道却不说:通过语言化思维监测医疗 LLM
What the model knows but does not say: monitoring medical LLMs through verbalized thoughts
Frederik Hauke · Jan Clusmann · Hanna Kreutzer · Christiane Kuhl · Tianyu Han · Felix Barajas Ordonez · et al.
中文摘要
维持临床语言模型安全的努力集中在模型的输入与输出上,但被操纵的模型可以返回看似完全正常的答案。本研究表明,模型自身以自然语言呈现的内部状态能暴露输出所掩盖的操纵。使用自然语言自编码器(autoencoder),将多模态 LLM(Gemma-3-27B)的激活翻译为文本,并让一个输入盲判(input-blind judge)分别阅读这一语言化的内部状态(思维通道)与模型答案。在两个临床攻击目标(遗漏病理、隐蔽推荐药物)、两种输入模态和两种投递机制下,思维通道在全部八种注入条件下比答案更频繁地捕获注入(p<0.05),而两个通道在干净输入上的假阳性率均接近零;遗漏结果在 Llama-3.3-70B 上同样成立。思维通道还在模型仍在阅读不安全临床请求、尚未决定是否拒绝之前,就将其标记为有害。以上均无需重新训练:现成的语言化器(verbalizer)与通用判读器(judge)即可。
关键要点
- 01问题:现有安全监测只看输入输出,被操纵模型可输出看似正常的答案而掩盖内部异常。
- 02方法:用自然语言自编码器将 Gemma-3-27B 激活转为文本,由输入盲判分别读取思维通道与答案。
- 03结果:八种注入条件下思维通道检出率显著高于答案(p<0.05),干净输入假阳性率近零。
- 04结果:思维通道在模型阅读不安全请求时即标记有害,早于其决定是否拒绝。
- 05局限:结论主要基于 Gemma-3-27B,遗漏结果在 Llama-3.3-70B 上复现。
解读
尚无解读。
原始英文摘要
arXiv:2611.24994v1 Announce Type: new Abstract: Efforts to keep clinical language models safe have concentrated on what enters and leaves the model, but a manipulated model can return an answer that looks entirely ordinary. Here we show that the model's own internal state, rendered in plain language, exposes manipulations the output conceals. Using a natural-language autoencoder, we translate the activations of a multimodal LLM (Gemma-3-27B) into text and let an input-blind judge read this verbalized internal state (the thoughts channel) and the model's answer separately. Across two clinical attack objectives (omitting a pathology, covertly recommending a drug), two input modalities, and two delivery mechanisms, the thoughts channel caught the injection significantly more often than the answer in all eight injected conditions (p<0.05), while both channels held near-zero false-positive rates on clean inputs; the omission result carried over to Llama-3.3-70B. The thoughts channel also registered unsafe clinical requests as harmful while the model was still reading them, before it decided whether to refuse. None of this requires retraining: an off-the-shelf verbalizer and a general-purpose judge are enough.