通过可审计的认知探索引导自动化科学发现
Steering Automated Scientific Discovery via Audited Epistemic Exploration
Fangxu Yu · Tao Feng · Xuying Ning · Haolin Yang · Jiaxuan You · Tianyi Zhou
中文摘要
科学发现需要根据新证据重新审视假设,并识别尚未探索的有前景方向。然而,研究智能体常提出缺乏观察支持的断言、探索的假设空间狭窄,且在识别到冲突证据时仍坚持原有信念。这些失败促使研究者寻找一种显式的认知状态,同时刻画对已有假设的信念与进一步探索的方向。为此,提出 Epistemic Compass(EpiCompass),一种即插即用的脚手架(scaffold),通过连接假设、证据、信念与修正的认知树(epistemic tree)维护该状态。随着研究推进,EpiCompass 依据累积证据重新评估已探索的假设,并利用未解决的冲突与未探索方向引导假设生成。它按预期认知增益(epistemic gain)选择候选,兼顾已有假设的细化与假设空间的扩展,并依据累积证据验证最终结论。在三个科学发现基准与三个 LLM 骨干上的实验表明,EpiCompass 持续优于基线,进一步分析显示其减少了无支持断言错误与信念修正失败。
关键要点
- 01问题:研究智能体常提出无证据支持的断言、探索空间狭窄且不修正信念。
- 02方法:EpiCompass 用认知树维护假设、证据、信念与修正的显式状态。
- 03方法:按预期认知增益选择候选,兼顾假设细化与假设空间扩展。
- 04结果:在三个科学发现基准和三个 LLM 骨干上持续优于基线。
- 05结果:分析显示无支持断言错误与信念修正失败均减少。
解读
尚无解读。
原始英文摘要
arXiv:2611.25214v1 Announce Type: cross Abstract: Scientific discovery requires revisiting the hypotheses in light of new evidence and identifying promising directions beyond those already explored. However, research agents often make claims that are not supported by their observations, explore a narrow hypothesis space, and retain beliefs despite recognizing conflicting evidence. These failures motivate the search for an explicit epistemic state that captures both beliefs about existing hypotheses and directions for further exploration. To this end, we propose Epistemic Compass (EpiCompass), a plug-and-play scaffold that maintains this state through an epistemic tree linking hypotheses, evidence, beliefs, and revisions. As an investigation progresses, EpiCompass reassesses the hypotheses explored against accumulated evidence and uses unresolved conflicts and unexplored directions to guide hypothesis generation. It selects candidates by anticipated epistemic gain, accounting for both refinement of existing hypotheses and expansion of the hypothesis space, and verifies the final conclusion against the accumulated evidence. Experimental results across three scientific discovery benchmarks and three LLM backbones demonstrate that EpiCompass consistently outperforms baselines, with further analyses showing reductions in errors of unsupported claims and failures to revise beliefs.