Git 未记录的内容:两个 AI 辅助研究仓库的来源可读性
What Git Does Not Record: Provenance Legibility in Two AI-Assisted Research Repositories
Koutian Wu · Zesen Huang · Hejia Geng
中文摘要
出版商与资助方日益要求作者披露 AI 工具对研究的贡献。对于计算研究,作者或审计者通常会查看代码仓库,但仓库实际能提供哪些信息仍不明确。研究审计了同一作者在同一领域的两个单人研究仓库,分别包含规则文件的规定性轨迹和运行中代理草稿本的叙述性轨迹,规模为 466 次和 103 次提交。两者可读性表现相反:规定性轨迹曾提交行中的 85.6% 在当前版本可见,但常设规则只保留结论并丢弃过程;叙述性轨迹记录过程,却仅显示 21.8%。按版本计算差距明显缩小:规定性轨迹 41 个版本中有 15 个无可见行,叙述性轨迹则有 18 个。自动审计读取的 Co-Authored-By 字段,是 Project A 中三个 AI trailer 键之一,覆盖带有任意此类键的 121 次提交中的 66 次。Project B 中,该字段相对明确的仓库内代理痕迹敏感度较低:39 次提交修改了用于保存代理工作笔记的文件,仅 1 次(2.6%)带有该字段。Project A 的仓库级比例也不稳定,在未预先设定的边界处从 2.7% 跃升至 17.7%。仓库渠道彼此不一致,两项比例均不能解释为 AI 参与率;其成因未被单独分离。研究逐项说明两类轨迹能够、不能及会错误确信支持的来源声明,构建可审计证据账本,并归纳实时记录研究来源的系统要求。
关键要点
- 01审计两个单人研究仓库,分别考察常设规则轨迹与代理草稿本轨迹。
- 02当前版本可见比例分别为 85.6% 与 21.8%,呈现相反的可读性特征。
- 03Co-Authored-By 字段在两个项目中的覆盖率与敏感度差异明显。
- 04仓库级 AI 比例会随未预设边界从 2.7% 跃升至 17.7%。
- 05仓库痕迹不能直接作为 AI 参与率,需明确其可支持的来源声明。
解读
尚无解读。
原始英文摘要
arXiv:2610.28655v1 Announce Type: cross Abstract: Publishers and funders increasingly ask authors to disclose how AI tools contributed to a piece of research. In computational work the repository is where an author or an auditor would look for the answer, and we ask what it can actually supply. We audit two single-investigator repositories by the same author in the same field, each carrying a different class of agent residue: a prescriptive trace of standing rule files and a narrative trace of a running agent scratchpad, in repositories of 466 and 103 commits. The two are legible in opposite ways. 85.6% of the lines ever committed to the prescriptive trace are visible at the current revision, and a standing rule states a conclusion and discards the episode behind it; the narrative trace records episodes and shows 21.8%. Counted by revision the gap is far smaller: 15 of the prescriptive trace's 41 revisions leave no visible line, against 18 of the narrative trace's 41. The Co-Authored-By trailer, the field an automated audit reads, is one of three AI trailer keys Project A carries and covers 66 of the 121 commits carrying any. In Project B it has low sensitivity against an explicit within-repository proxy: 39 commits revise a file maintained to hold an agent's working notes and 1 (2.6%) carries the trailer. Project A's repository-level rate is unstable in time too, stepping from 2.7% to 17.7% across a boundary we did not specify in advance. Repository channels disagree, and neither rate can be read as an involvement rate; we do not isolate the cause, and say so. We give a field-by-field account of which provenance claims such traces support, which they cannot, and which they answer confidently and wrongly, construct an auditable evidence ledger, and derive the requirements this places on any system meant to record research provenance as it happens.