cs.HC提交于 2026-10-19已译

PersonaCUABench: 评估服务多样化用户画像的计算机使用智能体基准

PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas

Haoming Li · Defu Cao · Xu Hu · Yingtao Luo · Yiyang Wang · Xianghe Xu · et al.

中文摘要

计算机使用智能体（CUA）通常在固定任务规范下评估，但用户可能遗漏需求、添加约束或在关键操作前要求审批。仅完成基准指令不能保证满足提问者需求。PersonaCUABench 固定 11 个领域的 74 项任务，仅改变提问者。扮演 4 组中 44 种画像之一的画像智能体，以用户口吻重写指令并回答 CUA 提问，CUA 仅见对话，不见画像或原始指令。在 9 个骨干模型（3,330 次运行）中，平均任务完成率较原始请求下降 36%。损失源于两方面：59.9% 的失分单元中，智能体处理了用户请求但仍出错；11.2% 是用户要求的停止被检查器判为失败。即使完全完成画像请求，用户给出最高满意度评分的比例仅为 62.7%，而原始请求为 84.1%。因此，CUA 评估不仅需关注任务完成，还需考察是否满足用户表达的要求与约束。

关键要点

01现有 CUA 评估基于固定任务规范，忽略了用户个性化需求、约束及审批要求带来的差异。
02提出 PersonaCUABench，固定 74 项任务，通过 44 种用户画像重写指令，测试 CUA 在仅见对话时的表现。
03在 9 个骨干模型上，引入用户画像后平均任务完成率较原始请求下降 36%。
04失分主要源于智能体处理请求出错（59.9%）及用户要求的停止被误判（11.2%）。
05即使任务完成，画像请求的用户最高满意度评分比例（62.7%）显著低于原始请求（84.1%）。

解读

尚无解读。

原始英文摘要

arXiv:2610.22990v1 Announce Type: cross Abstract: Computer-use agents (CUAs) are commonly evaluated against fixed task specifications, but users may omit requirements, add constraints, or require approval before consequential actions. Completing the benchmark instruction does not guarantee serving the person who asked. We introduce PersonaCUABench, which holds 74 tasks across 11 domains fixed and varies only who asks. A persona agent, playing one of 44 personas in four groups, rewrites each instruction in that user's own words and answers the CUA's questions, while the CUA sees only the dialogue, never the profile or the original instruction. Across nine backbones (3,330 runs), mean task completion falls by 36% relative to the original requests. The losses have two sources: in 59.9% of cells that lose points, the agent engages with what the user asked for and still gets it wrong, and 11.2% are stops the user requested that the checker scores as failures. Even when a persona request is fully completed, the user gives the top satisfaction rating in only 62.7% of cells, against 84.1% for the original requests. CUAs should therefore be evaluated not only on task completion but also on whether they meet the requirements and constraints users express. Code: https://github.com/MatrAIx-ai/PersonaCUABench. Project page: https://matraix.ai/research/persona-cua.html