跳到正文
返回论文列表
cs.AI提交于 已译

LoRA 领域适配中的推理监督与能力保持

Reasoning Supervision and Capability Retention in LoRA Domain Adaptation

Saleh Ashkboos · Dhruv Desai · Lavinia Ghita

中文摘要

在金融数据上微调基础模型有广泛的应用,包括从新闻中识别事件类型、监控风险与合规语言,以及回答监管文件与财报电话会议相关问题。此场景的主要挑战是在提升任务性能的同时不降低模型的泛化能力。本文对使用低秩适配(LoRA)微调 Qwen3-8B 进行了大量消融实验,比较三种训练方案:(1) 仅响应 LoRA,监督响应仅为标签本身;(2) 单适配器推理,将教师模型的推理轨迹与标签一同纳入训练损失;(3) 双适配器复合训练,分别用轨迹和标签训练独立的 LoRA 适配器。仅响应 LoRA 提升域内性能,并使通用能力接近冻结模型;域内变化在校准和 argmax 准确率上最大。单适配器推理下,域内指标仍落后于仅响应方案,而 MMLU-Pro 可略有上升,其他保持性基准在正常波动范围内。双适配器复合训练处于同一比较侧,未带来进一步增益。最后,在训练前对教师轨迹进行摘要压缩,表明更短的摘要使校准向仅响应方案靠拢但未达到该水平。

关键要点

  1. 01金融领域微调需在提升任务性能的同时保持模型泛化能力。
  2. 02仅响应 LoRA 提升域内性能,通用能力接近冻结模型。
  3. 03单适配器推理域内指标落后,MMLU-Pro 略升,保持性基准正常波动。
  4. 04双适配器复合训练无额外增益。
  5. 05摘要压缩教师轨迹使校准向仅响应方案靠拢但未达到。

解读

尚无解读。

原始英文摘要

arXiv:2611.17072v1 Announce Type: new Abstract: Fine-tuning foundation models on financial data has a wide range of applications, including identifying event types from news, monitoring risk and compliance language, and answering questions over filings and earnings calls. In this setting, the main challenge is to improve task performance without degrading the generalization capabilities of the model. In this paper, we present an extensive set of ablations for fine-tuning Qwen3-8B with low-rank adaptation (LoRA). We compare three training regimes: (1) response-only LoRA, where the supervised response is the label alone; (2) single-adapter reasoning, where reasoning traces from a teacher model are included alongside the labels in the training loss; and (3) dual-adapter composite training, where separate LoRA adapters are trained on traces and labels. Response-only LoRA improves in-domain performance and leaves general capabilities near those of the frozen model. The in-domain change is largest for calibration and argmax accuracy. Under single-adapter reasoning, the in-domain metrics remain behind those of the response-only regime, while MMLU-Pro can rise slightly and the other retention benchmarks stay within ordinary variation. Dual-adapter composite training remains on the same side of this comparison and yields no further gain. Finally, we compress the teacher traces by summarizing them before training, and show that shorter summaries move calibration toward the response-only regime without reaching it.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

AWS Machine Learning Blog68

Amazon SageMaker AI 推出训练会话 API 与训练 cookbook

Amazon SageMaker AI 推出 training session APIs,允许开发者在本地 Python 流程中编写训练循环,由服务托管 GPU、模型与 LoRA 加载、分布式执行、故障恢复和检查点。

推荐理由:文章拆解了训练会话 API、GRPO 循环和 Code RL 配方,并给出公开示例与 cookbook 入口,便于评估自定义训练循环的实现方式。