cs.CL提交于 2026-10-09已译

大规模语法概念标注：经微调部署的小型语言模型优于提示词驱动的前沿模型

Grammar Concept Annotation at Scale: Deployed Fine-Tuned Small Language Models Outperform Prompted Frontier Models

Marjan Celikik · Ana Peleteiro Ramallo · Javier Morales

中文摘要

纠错反馈（corrective feedback）是二语习得中证据最充分的促进因素之一，但课堂中给出的纠错很少能汇聚成关于语法掌握程度的、可指导行动的整体视图。提示词驱动的（prompted）前沿模型能从学习者与教师的课堂转写文本中提取该视图，但在规模化场景下成本高昂。我们通过在经过过滤与再平衡的、由教师生成的监督数据上微调 Qwen3.5 小型语言模型（SLM），并将一款高效的 0.8B 模型端到端部署到我方平台面向全体英语学习者的语法掌握跟踪系统中，从而填补了这一缺口。将标注契约内化进适配器权重后，0.8B 模型得以配以简短且匹配的提示，而非冗长的指令。在两个人工策展的基准上，无论已部署的 0.8B 模型还是作为参照的 4B 模型，在逐级加严的嵌套匹配准则——概念、证据片段（evidence span）、正确性——下，其精度与召回均优于提示词驱动的 GPT-5.4 与 GPT-5.6 Sol。已部署的 0.8B SLM 将服务成本降低约 16 倍。特征级在线实验显示，学习者参与度显著提升（+15.8%），关键业务指标亦有改善，包括排课时长（+2.1%）以及新课成交额（GMV）（+13.2%）。

关键要点

01问题：纠错反馈虽能促进二语习得，但课堂纠错难以汇聚为可指导行动的语法掌握视图；提示词驱动的前沿模型规模化成本高昂。
02方法：在经过滤与再平衡的、由教师生成的监督数据上微调 Qwen3.5 小型语言模型，将 0.8B 模型连同简短匹配提示部署为端到端语法掌握跟踪系统。
03结果：在两个人工策展基准的逐级加严嵌套匹配准则下，0.8B 部署模型与 4B 参照模型的精度与召回均超过 GPT-5.4 与 GPT-5.6 Sol。
04效果：服务成本降低约 16 倍，在线实验显示学习者参与度提升 15.8%，排课时长提升 2.1%，新课成交额提升 13.2%。

解读

尚无解读。

原始英文摘要

arXiv:2610.10827v1 Announce Type: new Abstract: Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate into an actionable view of grammar mastery. Prompted frontier models can provide such a view from learner--tutor lesson transcripts, but they are costly at scale. We close this gap by fine-tuning Qwen3.5 small language models (SLMs) on filtered and rebalanced teacher-generated supervision, then deploying an efficient 0.8B model in an end-to-end grammar mastery tracker for all English learners on our platform. Internalizing the annotation contract into adapter weights enables pairing the 0.8B model with a compact matched prompt rather than verbose instructions. On two human-curated benchmarks, both the deployed 0.8B model and a 4B reference comparator outperform prompted GPT-5.4 and GPT-5.6 Sol in precision and recall under nested matching criteria of increasing strictness: concept, evidence span, and correctness. The deployed 0.8B SLM reduces serving cost by approximately 16$\times$. A feature-level online experiment shows significant gains in learner engagement ($+15.8\%$) and key business metrics, including scheduled hours ($+2.1\%$) and GMV from new lessons ($+13.2\%$).