FrontierPhysics：评估端到端物理研究中的智能体
FrontierPhysics: Evaluating Agents for End-to-End Physics Research
Bingran You · Wenjun Ke · Ria Rosenauer · Bo Liu · Wenbo Chen · Min Chen · et al.
中文摘要
AI 智能体（agent）在物理研究中扮演越来越重要的角色。现有物理基准只评估固定问题、孤立子任务或给定解的论文复现，很少同时检查研究质量与正确性。FrontierPhysics 是端到端基准，含 42 位物理学家基于自身研究设计的 56 个任务，覆盖理论、计算与实验物理九个领域。每个任务要求智能体检索文献、制定研究计划、开展研究与实验，并产出结构化输出与科学手稿。该基准结合确定性验证器与专家撰写的评分标准，评估智能体的研究是否正确且可发表。在 22 个模型、26 种配置中，没有配置通过超过其 168 次运行中的 46 次。对智能体轨迹的人工审查显示，前沿智能体的主要弱点是推理不够严谨和专家直觉有限，而非物理知识缺口或执行错误。
关键要点
- 01现有物理基准只测固定问题或复现，缺少对研究质量与正确性的联合评估
- 02构建 56 个端到端任务，由 42 位物理学家基于自身研究设计，覆盖九个领域
- 03结合确定性验证器与专家评分标准，判断研究是否可发表
- 0422 个模型 26 种配置中，最佳配置仅通过 168 次运行中的 46 次
- 05主要短板是推理严谨性不足与专家直觉有限，而非知识或执行错误
解读
尚无解读。
原始英文摘要
arXiv:2611.06910v1 Announce Type: new Abstract: AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.