PairCLIP-SWA：相似度引导平衡与随机权重平均的 OpenCLIP 通用深度伪造图像检测
PairCLIP-SWA: Similarity-Guided Balancing and Stochastic Weight Averaging for Generalized Deepfake Image Detection with OpenCLIP
Md. Nakib · Sudipto Sarkar · A. J. A. Jubayer Talukder · Mohammad Ariful Haque
中文摘要
深度伪造检测器几乎总是部署为在验证集上得分最高的单一检查点，但这种选择的可靠性很少被衡量。PairCLIP-SWA 是一个单骨干检测器，通过渐进式解冻模块，在由 CLIP 相似度引导的一对一子采样构建的类别平衡语料上，对 OpenCLIP ViT-L/14 视觉塔进行适配，并在权重空间中对微调轨迹取平均。在 IEEE SP Cup 2025（DF-Wild-Cup）基准上，采用与最强已发表三骨干集成相同的协议进行域内训练，权重平均检测器达到 97.04% 准确率、98.64% AU-ROC 和 4.07% 等错误率，对比值为 94.46%、97.60% 和 5.00%。训练日志显示了平均为何重要：在平均的 epoch 上，非平均检查点的验证准确率波动达 5.34 个百分点，验证损失增加超过一倍，而平均值从不更差，最多好 5.99 个百分点。在 OpenForensics 衍生的基准上，同样的不稳定性变成错误排序：验证准确率选出的检查点（98.43%）在不相交的 10,905 张图像测试分区上得分为 88.85%，拒绝了 21.1% 的真实照片，而终端平均值在验证集上排名最后，得分为 94.76%。在相同 DF-Wild-Cup 划分上对 22 个内部配置（包括预处理消融）的研究表明，CLIP 预训练而非集成驱动准确率，二次人脸裁剪损失约 34 个百分点，频域输入迁移效果差。建议采用从早停划分测量的按类选择标准。
关键要点
- 01单检查点部署的可靠性很少被衡量，验证集选出的检查点可能严重误排。
- 02在权重空间平均微调轨迹，DF-Wild-Cup 上达 97.04% 准确率、4.07% 等错误率。
- 03非平均检查点验证准确率波动 5.34 点、损失翻倍，平均值从不更差且最多好 5.99 点。
- 04OpenForensics 衍生测试中验证选出的检查点仅 88.85%，终端平均达 94.76%。
- 05CLIP 预训练而非集成驱动准确率，二次人脸裁剪损失约 34 点，频域输入迁移差。
解读
尚无解读。
原始英文摘要
arXiv:2610.37874v1 Announce Type: cross Abstract: Deepfake detectors are almost always deployed as the single checkpoint that scored best on a validation split, yet the reliability of that choice is rarely measured. We measure it with PairCLIP-SWA, a single-backbone detector that adapts an OpenCLIP ViT-L/14 visual tower by progressive block unfreezing on a class-balanced corpus built by CLIP-similarity-guided one-to-one subsampling, and averages the fine-tuning trajectory in weight space. Trained in-domain on the IEEE SP Cup 2025 (DF-Wild-Cup) benchmark under the same protocol as the strongest published three-backbone ensemble, the weight-averaged detector reaches 97.04% accuracy, 98.64% AU-ROC and a 4.07% equal error rate, against 94.46%, 97.60% and 5.00%. Its training log shows why averaging matters: over the averaged epochs the non-averaged checkpoint swings across 5.34 points of validation accuracy while its validation loss more than doubles, whereas the average is never worse and is up to 5.99 points better. On an OpenForensics-derived benchmark the same instability becomes a misranking: the checkpoint that validation accuracy selects (98.43%) scores 88.85% on a disjoint 10,905-image test partition, rejecting 21.1% of genuine photographs, while the terminal average, ranked last on validation, scores 94.76%. A study of twenty-two in-house configurations on the identical DF-Wild-Cup split, including preprocessing ablations, shows that CLIP pre-training rather than ensembling drives accuracy, that secondary face cropping costs about 34 points, and that frequency-domain inputs transfer poorly. We recommend per-class selection criteria measured off the early-stopping split.