cs.CL提交于 2026-10-23已译

VEILED-SH：通过间接语言表达评估脆弱性（自伤）

VEILED-SH: Vulnerability Expression through Indirect Language Evaluation Dataset (Self-Harm)

Leanne Tan · Keith Chua · Shaun Khoo · Onno P. Kampman · Lim Ruling Charmaine · Hazirah Hoosainsah · et al.

中文摘要

AI 系统可能遗漏间接、文化情境化的痛苦表达，或提供与应用场景不匹配的支持。现有 benchmark 未同时处理文化差异、应用情境和从业者分歧。VEILED-SH 是用于评估对自杀、自伤及痛苦间接表达进行单轮回应的 benchmark 与构建方法。其 10,003 条 prompts 涵盖咨询聊天、论坛帖子、平台搜索和分诊取材，来源以新加坡为背景并结合公共数据集。经专家验证的多标签分类体系区分意图、交际功能和回应行为。六种自动 profile 分别依据三位同伴支持从业者和三位临床医生校准，并保留独立的参考策略用于代理对齐评分。对五个系统进行覆盖度增强样本评估后发现，即使系统回应相同，其在不同 profile 生成的参考策略下仍呈现代理对齐差异。

关键要点

01现有 benchmark 未同时涵盖文化、应用情境与从业者分歧
02VEILED-SH 含 10,003 条自杀、自伤及痛苦间接表达 prompts
03数据覆盖咨询聊天、论坛帖子、平台搜索和分诊场景
04六种 profile 由支持从业者与临床医生分别校准
05五个系统的代理对齐会随参考策略变化

解读

尚无解读。

原始英文摘要

arXiv:2610.26190v2 Announce Type: replace Abstract: AI systems may miss indirect, culturally situated distress or provide support mismatched to the application context. Existing benchmarks do not jointly address cultural variation, application context, and practitioner disagreement. We introduce VEILED-SH, a benchmark and construction methodology for evaluating single-turn responses to indirect expressions of suicide, self-harm, and distress. Its 10,003 prompts span counselor chat, forum posts, platform search, and triage, drawing on Singapore-grounded sources and public datasets. An expert-validated multi-label taxonomy distinguishes intent, communicative function, and response behavior. Six automated profiles, individually calibrated to three peer-support practitioners and three clinicians, retain separate reference policies for proxy-alignment scoring. Evaluating five systems on a coverage-enriched sample reveals variation in proxy alignment across profile-generated reference policies, even for the same system responses.