cs.AR提交于 2026-10-05已译

跨上下文KV-Cache重用的预算缓存修复

Budgeted Cache Repair for Cross-Context KV-Cache Reuse

Haeyong Kang · Chang D. Yoo

中文摘要

跨上下文KV-cache重用是在新前缀下预测共享段的keys和values而非重新计算，报告称可无质量损失实现。我们发现并非如此，并识别出两个问题。(1)隐藏成本：在MMLU和GSM8K上，重用造成显著精度损失。(2)决策单位错误：没有规则决定是否重用缓存来消除该成本。真正有帮助的是选择缓存的哪些部分重新计算，且选择的价值随选择单位增大而下降：单行（一个token的keys和values）的信息选择比随机多消除49.5%的缓存错误，64-token块消除10.6%，整个调用级别则nothing。预算缓存修复(BCR)在选择仍有价值的单位上操作。它从组装的缓存中草拟两个token，按这些token对缓存行的注意力对行排序，并精确重新计算固定预算的行，采用三种布局之一。成本是被支付而非预测消除，作为门控失败的草稿作为选择器成功。BCR将GSM8K恢复到密集预填充精度，同时仍从缓存服务大多数调用，其最佳布局在参考网格中优于每个重用基准的平均值。草稿还在相同预算下击败抛硬币选择器——这是先前评估缺乏的对照。

关键要点

01问题：跨上下文KV-cache重用存在隐藏精度损失，在MMLU和GSM8K基准上显著降低模型准确率
02发现：缓存选择的价值随决策单位增大而下降，单token行级别选择可消除49.5%错误，64-token块消除10.6%，整体调用级别无效
03方法BCR：在仍有选择价值的最小单位上操作，草拟两个token按注意力权重排序缓存行，精确重算固定预算的行，有三种布局
04结果：BCR在GSM8K上恢复到密集预填充精度，同时保持大多数调用从缓存服务，最佳布局优于所有重用基准
05对照实验：草稿选择器在相同预算下优于抛硬币随机选择，填补了先前评估缺乏的控制实验空白

解读

尚无解读。

原始英文摘要

arXiv:2610.02233v1 Announce Type: cross Abstract: Cross-context KV-cache reuse predicts a shared segment's keys and values under a new prefix instead of recomputing them, and has been reported to do so without quality loss. We find otherwise, and identify two problems. (1) A hidden cost: on MMLU and GSM8K, reuse costs substantial accuracy. (2) A decision at the wrong unit: no rule for deciding whether to reuse a cache removes that cost. What does help is choosing which parts of the cache to recompute, and the value of choosing well falls as the unit of choice grows: informed selection removes 49.5% of the cache error beyond chance at single rows (one token's keys and values), 10.6% at 64-token chunks, and nothing at the level of whole calls. Budgeted Cache Repair (BCR) acts at the unit where selection still pays. It drafts two tokens from the assembled cache, ranks cache rows by the attention those tokens pay them, and recomputes a fixed budget of rows exactly, in one of three layouts. The cost is paid rather than predicted away, and the draft that fails as a gate succeeds as a selector. BCR restores GSM8K to dense-prefill accuracy while still serving most calls from cache, and its best layout outperforms every reuse baseline's mean in the reference grid. The draft also beats a coin-flip selector at the same budget - a control prior evaluations lack.