cs.CL提交于 2026-10-23已译

TutorCaseBench：评估 LLM 导师支持学生应对挑战性行为的案例驱动基准

TutorCaseBench: A Case-Grounded Benchmark for Evaluating LLM Tutors in Supporting Students with Challenging Behaviors

Yeon Su Park · Chanyoung Park · Tae Soo Kim · Seungju Kim · Hyunwoo Kim · Daho Jung · et al.

中文摘要

大语言模型(LLM)日益被用作 AI 导师，但现有基准忽视了学习与行为困难学生常见的挑战性行为。TutorCaseBench 以教育学、教育心理学和特殊教育中的定性案例研究为基础，评估 AI 导师。其流程将文献记录的学生类别转化为特定案例情境、导师—学生对话，以及由来源证据支撑的评分标准。基准包含 5 类挑战下的 923 段对话，通过开放式回答生成（1,560 项）和多项选择（2,537 项）进行评测。在 11 个先进 LLM 中，即使最强模型也仅能满足 55.8% 的开放式案例评分标准；相比生成情境化回答，模型更擅长选择适当策略。结果显示，策略识别与情境化适应之间存在差距，开放权重模型尤其明显。所有数据已公开。

关键要点

01现有基准忽视学习与行为困难学生的挑战性行为
02以教育学、教育心理学和特殊教育案例研究构建评测
03涵盖 5 类挑战、923 段对话及两类评测任务
04最强模型仅满足 55.8% 的开放式案例评分标准
05模型选择策略的能力强于生成情境化回答

解读

尚无解读。

原始英文摘要

arXiv:2610.30828v1 Announce Type: new Abstract: LLMs are increasingly used as AI tutors, yet existing benchmarks overlook the challenging behaviors common among students with learning and behavioral difficulties. We introduce TutorCaseBench, a benchmark that grounds AI-tutor evaluation in qualitative case studies from pedagogy, educational psychology, and special education. Our pipeline converts documented student categories into case-specific scenarios, tutor-student dialogues, and rubrics grounded in source evidence. The benchmark contains 923 dialogues across five challenge types, evaluated through open-ended response generation (1,560 items) and multiple-choice selection (2,537 items). Across 11 state-of-the-art LLMs, even the strongest model satisfies a rubric criterion in only 55.8% of open-ended cases, while models are better at selecting appropriate strategies than generating situated responses. These results reveal a gap between intervention recognition and context-specific adaptation, especially for open-weight models. All data are publicly available at https://github.com/kixlab/TutorCaseBench.