CompressionBench：编码智能体如何压缩 LLM？
CompressionBench: How can your coding agents compress LLMs?
Xinrui Zhong · Zhifeng Li · Zhiming Wang · Yongqi Liu · Shanglin Yang · Yang Zhou · et al.
中文摘要
递归自我改进要求 AI 系统改进后续系统所用的模型与基础设施。LLM 压缩提供了一个具体的效率目标：在更小的模型存储预算内保留预训练模型的行为。研究编码智能体能否自主完成这一步。将压缩形式化为终端任务：智能体接收源检查点、存储限制、推理环境与时间预算，然后构造压缩权重与可执行推理代码。智能体控制其算法与实验；困惑度反馈引导搜索，独立验证检查最终模型。在六个智能体与六种模型—规模设置下，36 份提交中 29 份通过验证。按任务选择困惑度最低的智能体设计，平均困惑度较 BF16 上升 3.4%，下游准确率 77.33%，而 BF16 为 79.56%。记录实验揭示智能体如何取得这些结果以及改进在何处失效。智能体在 24 次修订中的 21 次于存储几乎不变时降低 PPL；完整模型反馈逆转了 79 个先前局部排名中的 17 个；生成与推理测试暴露了仅靠似然遗漏的失败。结果指出表示设计、实验判断与能力验证是智能体驱动模型压缩的核心，也是面向效率的自我改进的可度量组成部分。
关键要点
- 01问题：递归自我改进需要 AI 自主压缩 LLM，但编码智能体能否胜任尚不明确。
- 02方法：将压缩设为终端任务，智能体在存储与时间预算内构造压缩权重和推理代码。
- 03结果：36 份提交中 29 份通过验证，最优设计平均困惑度仅升 3.4%，下游准确率 77.33%。
- 04发现：完整模型反馈逆转 79 个局部排名中的 17 个，生成与推理测试暴露似然遗漏的失败。
- 05局限：表示设计、实验判断与能力验证仍是智能体驱动压缩的瓶颈。
解读
尚无解读。
原始英文摘要
arXiv:2611.06858v1 Announce Type: new Abstract: Recursive self-improvement requires AI systems to improve the models and infrastructure used by subsequent systems. LLM compression offers a concrete efficiency objective: retain a pretrained model's behavior within a smaller model storage budget. We study whether coding agents can carry out this step autonomously. We formulate compression as a terminal task in which an agent receives a source checkpoint, storage limit, inference environment, and time budget, then constructs compressed weights and executable inference code. The agent controls its algorithm and experiments; perplexity feedback guides search, and independent verification checks the final model. Across six agents and six model--size settings, 29 of 36 submissions pass verification. Selecting the lowest-perplexity agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream accuracy, compared with 79.56% for BF16. Recorded experiments reveal how agents achieve these outcomes and where improvement breaks down. Agents lower PPL in 21 of 24 revisions at nearly fixed storage; complete-model feedback reverses 17 of 79 prior local rankings; and generation and inference tests expose failures that likelihood alone misses. The results identify representation design, experimental judgment, and capability validation as central to agent-driven model compression, a measurable component of efficiency-oriented self-improvement.