跳到正文
返回论文列表
cs.AI提交于 已译

AI4Math 淘金热 II:AI 辅助证明的特征是什么?

The Gold Rush in AI4Math, II: What are signatures of AI-assisted Proofs?

Yuanchuan Guo · Jiashun Jin · Zheng Tracy Ke · Bingcheng Sui

中文摘要

AI 在数学中的使用日益增多,引出一个问题:能否将 AI 对证明的参与与 AI 对写作的辅助区分开来并加以检测?历史 arXiv 论文、近期披露实质性使用 AI 的论文以及 OpenAI 发布的手稿,为研究该问题提供了丰富的实证基础。然而仍存在若干挑战:缺乏匹配的病例-对照对;披露可能不完整;缺乏可直接用于捕捉证明中 AI 参与信号的特征。为应对这些挑战,通过将现有手稿与使用生成式 AI 生成的手稿结合,构建了包含 8,281 篇论文的五个队列。这些队列覆盖不同 AI 参与程度,并包含针对同一研究问题的匹配论文对。随后引入 ProofTrace 框架,将每篇论文表示为直接证明支持图(GIPS),并提取可解释的数学组织特征。研究得出三项发现。第一,特征的 PCA 投影揭示出 AI 参与的定性梯度:AI 改写论文仍接近人类论文,两个 AI 生成队列沿大致共享的方向偏离,而披露实质性 AI 使用的论文处于中间位置,并随时间向 AI 生成队列偏移。第二,识别出 AI 生成证明的反复出现的特征,如外部引用结果更少、更依赖中间结论、图连通性改变;许多特征在研究领域、手稿长度或图规模控制下依然存在。最后,在独立检测实验中,ProofTrace 能区分人类撰写论文与披露实质性 AI 使用的论文,其性能在 AI 改写下保持稳定,而最先进的基于文本的检测器性能急剧下降。

关键要点

  1. 01构建 8,281 篇论文的五个队列,含匹配病例-对照对,覆盖不同 AI 参与程度。
  2. 02提出 ProofTrace 框架,将论文表示为直接证明支持图(GIPS)并提取可解释特征。
  3. 03发现 AI 参与呈定性梯度:AI 改写接近人类,AI 生成方向共享,披露实质性使用居中并随时间偏移。
  4. 04识别 AI 生成证明的反复特征:外部引用少、依赖中间结论、图连通性改变,且在控制变量下稳健。
  5. 05ProofTrace 检测性能在 AI 改写下稳定,优于基于文本的检测器。

解读

尚无解读。

原始英文摘要

arXiv:2611.08412v1 Announce Type: new Abstract: The growing use of AI in mathematics raises a question: can we detect AI involvement in proofs, separately from AI assistance with writing? Historical arXiv papers, recent papers disclosing substantive AI use, and manuscripts released by OpenAI provide a rich empirical basis for studying this question. However, several challenges remain: lack of matched case-control pairs; potential incompleteness of disclosure; and lack of ready-to-use features designed to capture signals of AI involvement in proofs. To address these challenges, we construct five cohorts comprising 8,281 papers by combining existing manuscripts with those we produced using generative AI. These cohorts span different levels of AI involvement and include matched pairs of papers addressing the same research problems. We then introduce ProofTrace, a framework that represents each paper as a graph of immediate proof support (GIPS) and extracts interpretable features of mathematical organization. Our study yields three discoveries. First, principal-component projections of our features reveal a qualitative gradient of AI involvement, where AI-reworded papers remain close to human papers, the two AI-generated cohorts depart in broadly shared directions, and papers disclosing substantive AI use occupy intermediate positions which shifts toward AI-generated cohorts over time. Second, we identify recurring signatures of AI-generated proofs, such as fewer externally cited results, greater reliance on intermediate conclusions, and altered graph connectivity; many of these signatures persist under controls for research field, manuscript length, or graph size. Finally, in a separate detection experiment, ProofTrace distinguishes human-written papers from those disclosing substantive AI use. Its performance remains stable under AI rewording, whereas the performance of state-of-the-art text-based detectors declines sharply.

同方向论文 · cs.AI

查看全部 →