cs.SE提交于 2026-10-28已译

Agent 还是 Human？在无作者签名情况下检测 AI 编码 Agent 的提交

Agent or Human? Detecting Commits by AI Coding Agents in the Absence of Authorship Signatures

Kosei Horikawa · Hao Li · Yutaro Kashiwa · Youssef Esseddiq Ouatiti · Mohammed Sayagh · Bram Adams · et al.

中文摘要

Claude Code、GitHub Copilot coding agent 和 Cursor 等 AI 编码 Agent 现已直接向开源仓库提交代码。现有研究通常依赖 bot 账号或 Co-authored-by 等作者签名来识别 Agent 提交，但这些签名是可选的、可被移除的，且常附在 Pull Request 而非单个 Commit 上。本研究基于 1,226 个仓库的 7,837,946 条提交，涵盖五种编码 Agent 和 Human 提交，探讨了无签名检测的信号与局限。在屏蔽签名后，检测器将 Agent 提交排在 Human 提交之前，ROC AUC 最高达 0.859，主要信号来自消息冗长度和变更大小，而非代码本身。在未微调的情况下，gpt-6-sol 达到了与 XGBoost 和 TF-IDF 相似的 AUC。然而，在观察到的 Agent 占比下，全量测试集上最佳检测器的精确率仅为 0.332。检测在保留的 Claude Code 和 Copilot 样本上迁移效果不佳，TF-IDF 对 Claude Code 的 AUC 仅为 0.524。时间变化因 Agent 而异，Copilot 的 AUC 先降后升，而更多训练历史有助于 Claude Code 但无助于 Copilot。结果表明，无签名检测可用于优先审查提交，但不能替代记录归属，依赖签名标签的研究应将其视为 Agent 活动量的下限。

关键要点

01现有研究依赖可选且可移除的作者签名识别 Agent 提交，导致无签名时难以区分 Agent 与 Human 提交。
02基于 7,837,946 条提交的研究发现，无签名检测的 ROC AUC 最高达 0.859，主要信号来自消息冗长度和变更大小。
03全量测试集上最佳检测器精确率仅为 0.332，且检测在保留的 Claude Code 和 Copilot 样本上迁移效果不佳。
04无签名检测可用于优先审查提交但不能替代记录归属，依赖签名标签的研究应将其视为 Agent 活动量的下限。

解读

尚无解读。

原始英文摘要

arXiv:2610.36229v1 Announce Type: cross Abstract: AI coding agents such as Claude Code, GitHub Copilot coding agent, and Cursor now commit directly to open-source repositories. Empirical studies of these agents identify their commits through authorship signatures, such as bot accounts and Co-authored-by trailers. These signatures are optional, can be removed, and are often attached to pull requests rather than to individual commits. Hence, it is unclear whether Agent commits can be told apart from Human commits once the signatures are gone. We investigate the signals and limits of signature-free detection using 7,837,946 commits from 1,226 repositories, covering five coding agents and Human commits. After signatures are masked, detectors rank Agent commits above Human commits with a ROC AUC of up to 0.859, and most of this signal comes from message verbosity and change size rather than from the code itself. On the shared sample, gpt-6-sol without fine-tuning reaches AUCs similar to XGBoost and TF-IDF. At the observed Agent share, however, the best detector on the full test set reaches a precision of only 0.332. Detection transfers poorly to held-out Claude Code and Copilot, with TF-IDF reaching an AUC of only 0.524 for Claude Code. Temporal changes also depend on the agent, e.g., Copilot AUC drops and later recovers, while more training history helps Claude Code but not Copilot. These results indicate that signature-free detection can prioritize commits for review but cannot replace recorded attribution, and that studies relying on signature-based labels should treat them as a lower bound on agent activity.