跳到正文
返回论文列表
stat.ME提交于 已译

自适应实验可掩盖模型误差:检测极限与证伪预算

Adaptive Experimentation Can Hide Model Error:Detection Limits and Falsification Budget

Ramnarayan Choudhary

中文摘要

实验可能对工作模型的参数有信息量,却无法揭示该模型是否错误。自适应设计通过反复选择错误模型拟合良好的设置,会加剧这种错配。研究以实验者自身的采样分布为基准,界定了任何有效充分性检验的效力上限,并区分了所选数据中缺失的信息与检测器丢失的信息。一种可计算的诊断指标,相对于工作模型声明的扩展,度量未解决的误差,并由此引出一种无需训练的规则:预留部分设计预算用于证伪。实证记录好坏参半:独立设定的主研究未通过检测资格,后续的分子诊断未达到其改进标准。在冻结规则于新的连续零面板上失败后,一项披露的次要视野校准修订在 T = 240 时检测出 35.8% 的误设定实验,而类别分离对照为 31.7%,但在 T = 480 时无已解决的优越性。在修正的 289 单元实测回放中,它优于随机设计,但落后于类别判别 EIG。一个条件性无效界解释了为何剩余时间重要,但未证明门限常数或其效力。这些结果支持将参数学习、模型检查与检测器性能分开;它们并未确立普遍优越的设计策略或前瞻性科学发现。

关键要点

  1. 01问题:自适应实验可能反复选择错误模型拟合良好的设置,从而掩盖模型误差。
  2. 02方法:以实验者采样分布界定充分性检验效力上限,并提出可计算诊断指标与预留证伪预算的免训练规则。
  3. 03结果:T=240 时检测 35.8% 误设定实验,对照为 31.7%;T=480 时无已解决优势。
  4. 04局限:主研究未通过检测资格,冻结规则在连续零面板上失败,未证明门限常数或普遍优越策略。

解读

尚无解读。

原始英文摘要

arXiv:2611.07529v1 Announce Type: cross Abstract: An experiment can be informative about the parameters of a working model and uninformative about whether that model is wrong. Adaptive design can reinforce this mismatch by repeatedly selecting settings where an incorrect model fits. We bound the power of any valid adequacy test in terms of the experimenter's own sampling distribution, then distinguish information absent from the selected data from information lost by the detector. A computable diagnostic measures unresolved error relative to a declared expansion of the working model. It motivates a training-free rule that reserves part of the design budget for falsification. The empirical record is mixed: the independently specified primary study fails detection eligibility, and a later numerator diagnostic fails its improvement criteria. After the frozen rule fails on fresh continuous-null panels, a disclosed secondary horizon-calibrated revision detects 35.8% of misspecified campaigns versus 31.7% for a class-separation control at T = 240, but has no resolved advantage at T = 480. In a corrected 289-unit measured replay it improves on random design and trails class-discrimination EIG. A conditional futility bound explains why remaining time matters without proving the gate constant or its power. These results support separating parameter learning, model checking and detector performance; they do not establish a universally superior design policy or prospective scientific discovery.

同方向论文 · stat.ME

查看全部 →