cs.HC提交于 2026-10-21已译

ValiAssist: 通过演示扩展 LLM 生成编程反馈的验证

ValiAssist: Scaling LLM-generated Programming Feedback Validation by Demonstration

Xiaohang Tang · Rehema Abulikemu · Peng Gao · Yan Chen

中文摘要

生成式 AI 能大规模创建编程反馈，但验证其与教师教学目标及策略的一致性仍具挑战性。教师难以预先完全指定验证标准，这些标准往往通过检查具体实例和修正不一致反馈而逐渐形成。ValiAssist 是一个基于演示的验证系统，它从教师对 AI 生成反馈的修改中推断可复用的验证逻辑，并允许教师在逻辑应用于其他学生提交时进行检查和细化。在涉及 16 名教师的对比研究中，ValiAssist 帮助教师创建和细化验证标准，产生的反馈与教学意图的一致性更高（91% 对 57%），并将修正问题标准的准确率提高了一倍以上（77% 对 32%）。研究结果表明，基于演示的验证能帮助教师将针对单个案例的判断转化为显式、可复用的逻辑，同时逐步澄清这些判断在哪些情况下适用，哪些情况下不适用。

关键要点

01针对教师难以预先明确 AI 反馈验证标准的问题，提出基于演示的验证方法。
02系统 ValiAssist 从教师对 AI 反馈的修改中推断可复用逻辑，并支持跨案例应用与细化。
03对比研究显示，使用 ValiAssist 后反馈与教学意图一致性达 91%，显著高于 57% 的基线。
04修正问题标准的准确率从 32% 提升至 77%，有效帮助教师将个案判断转化为通用逻辑。

解读

尚无解读。

原始英文摘要

arXiv:2610.27248v1 Announce Type: cross Abstract: Generative AI can create programming feedback at scale, but validating its alignment with instructors' pedagogical goals and strategies remains challenging. Instructors may struggle to fully specify validation criteria upfront, as these often emerge through inspecting concrete examples and revising misaligned feedback. We present ValiAssist, a validation-by-demonstration system that infers reusable validation logic from instructors' revisions of AI-generated feedback and allows instructors to inspect and refine that logic as it is applied across other student submissions. In a comparative study with 16 instructors, ValiAssist helped instructors create and refine validation criteria, producing feedback that better aligned with their pedagogical intent (91% vs. 57%) and more than doubling the accuracy of correcting problematic criteria (77% vs. 32%). Our findings suggest that validation by demonstration can help instructors turn judgments they know how to make on individual cases into explicit, reusable logic, while progressively clarifying where those judgments do and do not generalize.