ReproBridge：将可复现性构建到机器学习研究工件中
ReproBridge: Building Reproducibility into ML Research Artifacts
Yubo Wang · Rui Meng · Jaehyun Nam · Tomas Pfister · Jinsung Yoon
中文摘要
复现机器学习实验需要从论文、已发布代码和支持工件中分散的信息重建流程。对编码智能体（coding agent）而言，复现成功可能取决于其填补缺失细节的能力，而非仅依赖已发布工件。研究是否保留恢复的流程与决策能改善后续独立复现。定义标准化工件设计，包含精炼代码库、环境配置、实验启动器和复现指南，并引入 ReproBridge，一种构建这些精炼可复现工件的智能体工作流。ReproBridge-Bench 提供来自 36 篇论文的 387 个实验的 1,029 个任务，每个任务以五个人工验证的参考分数为锚点。智能体在新会话中尝试任务，使用原始代码或精炼工件；独立审计员（Auditor）检查执行证据。在四个评估模型中，精炼工件将平均复现成功率从 76.4% 提升至 92.6%，并将下游 API 支出平均降低 19.7%。跨模型成功率差距从 21.6 个百分点缩小至 3.9 个百分点，四个模型平均成功率均超过 90%。消融研究中移除任何组件都会降低成功率，其中代码和环境移除损失最大。这些结果表明，保留执行知识有助于原始代码成功率较低的模型更可靠且以更低 API 成本复现实验。
关键要点
- 01复现 ML 实验需从论文、代码和工件中重建流程，编码智能体需填补缺失细节。
- 02提出标准化工件设计（精炼代码库、环境、启动器、复现指南）与 ReproBridge 智能体工作流。
- 03ReproBridge-Bench 含 36 篇论文 387 个实验的 1,029 个任务，每任务五个参考分数。
- 04精炼工件将平均复现成功率从 76.4% 提至 92.6%，API 支出降 19.7%，跨模型差距从 21.6 缩至 3.9 个百分点。
- 05消融显示移除代码和环境组件损失最大，保留执行知识可提升低成功率模型的复现可靠性。
解读
尚无解读。
原始英文摘要
arXiv:2611.08452v1 Announce Type: new Abstract: Reproducing machine learning experiments requires reconstructing procedures from information scattered across papers, released code, and supporting artifacts. For coding agents, reproduction success can therefore depend on their ability to fill in missing details, rather than on the released artifacts alone. We ask whether preserving recovered procedures and decisions improves subsequent independent reproduction. We define a standardized artifact design comprising a refined codebase, environment setup, experiment launchers, and a reproduction guide, and introduce ReproBridge, an agent workflow for constructing these refined reproducible artifacts. ReproBridge-Bench provides 1,029 tasks from 387 experiments across 36 papers, anchored by five human-verified reference scores per task. Agents attempt tasks in fresh sessions with raw code or refined artifacts; a separate Auditor checks execution evidence. Across four evaluated models, refined artifacts raise mean reproduction success from 76.4% to 92.6% and reduce downstream API spending by 19.7% on average. The cross-model success-rate spread narrows from 21.6 to 3.9 percentage points, with all four models achieving mean success rates above 90%. Removing any component reduces success in the ablation study, with the largest losses from code and environment removal. These results suggest that preserving execution knowledge helps even models with lower raw-code success reproduce experiments more reliably and at lower API cost.