合并与分支：面向推荐的高效树结构 LLM 监督微调
Merge and Branch: Efficient Tree-Structured Supervised Fine-Tuning of LLMs for Recommendation
Yunqin Zhu · Yao Xie
中文摘要
基于 LLM 的生成式推荐需要读取长且文本丰富的交互历史，却只生成很短的语义标识符（SID）。这种不对称使监督微调效率低下：编码历史代价高昂，而每条历史通常只提供一个目标的监督信号。Merge and Branch（M&B）通过降低输入成本并让每条已编码历史提供更多学习信号来解决该问题。Merge 将每个历史条目的 SID 与文本压缩为少量可学习 token；Branch 则利用压缩后的历史预测多个目标 SID。为在共享历史计算的同时保持各目标独立，M&B 将每个样本组织为浅而宽的树结构，并开发 tree FlashAttention 高效执行该布局：共享历史只处理一次，所有分支的梯度均可回传以进行端到端训练。树结构 key/value 缓存进一步将历史共享扩展到受限解码。公开基准实验表明，M&B 在显著提升训练与解码效率的同时优于强推荐基线。
关键要点
- 01问题：LLM 推荐需编码长历史却只生成短 SID，监督微调输入成本高、每条历史仅对应一个目标。
- 02方法：Merge 将历史条目的 SID 与文本压缩为少量可学习 token，Branch 用压缩历史预测多个目标 SID。
- 03方法：以浅而宽的树组织样本，配合 tree FlashAttention 一次处理共享历史并回传所有分支梯度。
- 04结果：树结构 key/value 缓存将历史共享扩展到受限解码，公开基准上优于强基线且训练解码更高效。
解读
尚无解读。
原始英文摘要
arXiv:2610.37973v1 Announce Type: cross Abstract: LLM-based generative recommendation reads long, text-rich interaction histories but generates only short semantic identifiers (SIDs). This asymmetry makes supervised fine-tuning inefficient: encoding the history is expensive, yet each history usually provides supervision for only one target. We present Merge and Branch (M&B) to reduce input cost and learn more from each encoded history. Merge compresses each historical item's SID and text into a small number of learned tokens. Branch uses the compressed history to predict multiple target SIDs. To share history computation while keeping these targets independent, M&B organizes each example as a shallow, wide tree. We develop tree FlashAttention to execute this layout efficiently, processing the shared history once and propagating gradients from every branch for end-to-end training. A tree-structured key/value cache extends history sharing to constrained decoding. Experiments on public benchmarks show that M&B outperforms strong recommendation baselines while substantially improving training and decoding efficiency.