cs.CL提交于 2026-10-19已译

EKLAV: 仅基于答案监督的诱导关键思维链学习

EKLAV: Elicited Key CoT Learning via Answer only superVision

Adarsh Singh · Aman Singh · Kushal Raj Bhandari · Soham Dan · Anoop Kumar · Vivek Gupta

中文摘要

推理蒸馏通常训练学生模型复现教师的思维链（chain-of-thought）轨迹及最终答案。EKLAV 将教师推理轨迹作为特权训练上下文而非模仿目标，通过任务依赖的放置策略将部分或全部轨迹移至提示词中，仅对最终任务答案进行监督。在 BRIGHT 段落重排序任务中，EKLAV 在两个模型家族的四个检查点上均提升了平均 nDCG@10，在 48 次模型-领域对比中赢得 41 次，同时训练计算量减少最多 31.6%，推理时间减少最多 32.7%。在四个保留表格重排序基准上，EKLAV 在三项中表现最佳，平均比全轨迹蒸馏高 3.0 分，比无推理仅答案训练高 5.5 分。生成的轨迹分析显示，EKLAV 学生模型首次口头判断的相对位置更靠后（0.76 对 0.44），决策反转更少（5% 对 16%），且其推理对自身排名的预测性优于 STD-COT，但弱于基础模型。该方法复用固定教师语料，无需学生 rollout 或迭代教师反馈，适用于逐点（pointwise）和列表式（listwise）形式。

关键要点

01提出 EKLAV 方法，将教师思维链作为输入上下文而非模仿目标，仅监督最终答案。
02在 BRIGHT 重排序任务中，EKLAV 在 48 次对比中赢得 41 次，提升 nDCG@10 指标。
03相比全轨迹蒸馏，EKLAV 平均性能高 3.0 分，且训练计算量减少最多 31.6%。
04分析显示 EKLAV 学生模型决策反转率更低，推理对自身排名的预测性优于标准 CoT。
05方法无需学生 rollout 或迭代反馈，适用于逐点和列表式重排序形式。

解读

尚无解读。

原始英文摘要

arXiv:2607.25182v2 Announce Type: replace Abstract: Reasoning distillation trains a student to reproduce both the teacher's chain-of-thought trace and its final answer. We ask whether the trace is better used as privileged training context than as an imitation target. EKLAV keeps the teacher's reasoning in the input sequence but supervises only the final task answer, applying a task-dependent placement that moves part or all of the trace into the prompt. On BRIGHT passage reranking, EKLAV improves average nDCG@10 at all four tested checkpoints across two model families, winning 41 of 48 model-domain comparisons while reducing training compute by up to 31.6% and inference time by up to 32.7%. On four held-out table-reranking benchmarks, it is the best of four conditions on three, exceeding full-trace distillation by 3.0 points and answer-only training without reasoning by 5.5 points on average. Analysis of generated traces reveals that EKLAV students defer their first verbalized verdict later (0.76 vs. 0.44 relative position), reverse fewer decisions (5% vs. 16%), and produce reasoning more predictive of their own rankings than STD-COT's, though less predictive than the base model's. The recipe reuses a fixed teacher corpus, requires no student rollouts or iterative teacher feedback, and applies to both pointwise and listwise formulations. Code and Data are at https://github.com/AdarshSingh7647/Eklavya