绿色代码还是高代价代码?评估迭代式 LLM 代码优化的能耗
Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM-Based Code Optimizations
Sadra Emami · Benjamin Weigell · Mohammad Reza Mousavi · Gunel Jahangirova · Hugo Araujo · Bernhard Bauer
中文摘要
大型语言模型(LLM)正被用于优化代码性能,但这通常默认执行速度提升会自然降低总能耗。研究以 EffiBench 全部 1,000 道题评测 11 个开源模型及 4 种解码配置,并逐进程采集硬件级能耗。推理能耗为 0.79–8.68 MJ,约等于 250–2800 次 GPT-4.1 查询;限制最大输出 token 的影响最大,相比默认配置最多降低 69% 能耗。44 种配置中有 42 种生成的代码单题耗能高于标准人工参考实现,增幅为 +4%–+886%;其余两种仅有微弱收益,盈亏平衡点约为 10^6–10^7 次执行。混合专家(Mixture-of-Experts)架构的推理能耗显著低于同等规模稠密模型,且不牺牲正确性,表明架构选择比解码参数更能影响可持续性。结果显示,性能优化与能源效率并不总是一致:LLM 自我改进倾向于生成运行更快但能耗更高的代码。
关键要点
- 01检验 LLM 自我改进代码能否同时降低执行能耗及抵消优化成本
- 02在 EffiBench 全部 1,000 题上评测 11 个开源模型和 4 种解码配置
- 03推理能耗为 0.79–8.68 MJ;限制输出 token 最多可降低 69% 能耗
- 0444 种配置中 42 种单题耗能比人工参考实现高 4%–886%
- 05混合专家架构比同规模稠密模型更节能,且未降低正确性
解读
尚无解读。
原始英文摘要
arXiv:2610.32891v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used to optimize code for performance. This practice often assumes that improvements in execution speed naturally translate into overall lower energy consumption. We challenge this assumption empirically by measuring whether performance-optimized code produced by LLM self-refinement pipelines is also energy-efficient relative to canonical human reference solutions, and whether the energy cost of the optimization process can be offset by subsequent execution-time savings. We evaluate 11 open-source models across four decoding configurations on all 1,000 problems of EffiBench, a state-of-the-art performance benchmark, while collecting per-process, hardware-level energy measurements throughout. Inference energy spans 0.79 to 8.68 MJ across model-variant pairs, which is roughly equivalent to the energy of 250 to 2800 GPT-4.1 queries. Limiting the maximum number of output tokens has the largest impact, reducing energy use by up to 69% compared to the default configuration. On the execution side, 42 of 44 configurations produce code that consumes more energy per problem than the canonical human reference, with overheads ranging from +4% to +886%. The two exceptions achieve only marginal gains, with break-even points on the order of 10^6 to 10^7 executions. Mixture-of-Experts architectures consume substantially less inference energy than equivalently-sized dense models without sacrificing correctness, making architectural choice a more consequential sustainability lever than decoding parameters. Performance optimization and energy efficiency are not always aligned goals: LLM self-refinement consistently produces faster-seeking but energy-heavier code relative to what a human engineer would write. All artifacts are available at https://doi.org/10.5281/zenodo.23598170.