SpecCompact：用槽并行语义推测验证加速 LLM Agent 上下文压缩
SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification
Yulu An · Yifei Feng · Xiangyang Yin
中文摘要
LLM agent 通过上下文压缩（context compaction）管理不断增长的交互历史，但用大模型对长历史做摘要会带来显著延迟，因此压缩需要在信息保留与延迟之间权衡。观察到结构化摘要槽（slot）在独立生成时仍能保留信息，据此提出 SpecCompact 框架，通过槽并行语义推测（slot-parallel semantic speculation）加速压缩。该框架微调一个小型起草模型（drafter）并行生成各槽，目标模型逐行验证提议内容并修复被拒部分，后续起草使用更新后的槽前缀；系统优化进一步减少重复的上下文处理与模型调用。在 AppWorld 与 τ³ 两个基准、两种目标模型上，SpecCompact 相比 Hermes 式摘要取得 2.04–5.63× 的压缩加速。独立的下游评测显示性能权衡随模型与任务而异。Hermes 内的用户研究进一步显示中位压缩加速达 8.68×。
关键要点
- 01问题：大模型摘要长交互历史延迟高，压缩需在信息保留与延迟间权衡。
- 02方法：微调小型起草模型并行生成摘要槽，目标模型逐行验证并修复被拒内容。
- 03结果：AppWorld 与 τ³ 上相比 Hermes 式摘要加速 2.04–5.63×。
- 04结果：Hermes 用户研究中位压缩加速 8.68×。
- 05局限：下游性能权衡随模型与任务不同而变化。
解读
尚无解读。
原始英文摘要
arXiv:2611.06876v1 Announce Type: new Abstract: LLM agents use context compaction to manage growing interaction histories, but summarizing long histories with large models can introduce substantial delays. Compaction therefore requires balancing information retention against latency. We observe that structured summary slots can retain information when generated independently. Based on this observation, we present SpecCompact, a framework that accelerates compaction through slot-parallel semantic speculation. SpecCompact fine-tunes a small drafter and generates slots in parallel. The target model verifies each proposed line and repairs rejected content, while subsequent drafting uses the updated slot prefix. System optimizations further reduce repeated context processing and model calls. Across AppWorld and $\tau^3$ benchmarks with two target models, SpecCompact achieves $2.04$--$5.63\times$ compaction speedups over Hermes-style summarization. Separate downstream evaluations reveal model- and task-dependent performance trade-offs. A user study within Hermes further shows an $8.68\times$ median compaction speedup.