通用基础模型在化学性质预测任务上超越专用模型
General-purpose foundation models surpass specialized ones on chemical property prediction tasks
Stepan N. Ostarkov · Gregorii A. Semakin · Ivan A. Bespalov · Michael G. Medvedev
中文摘要
基础模型（foundation models, FMs）从大量无标注数据中学习可复用表征。化学领域的对应数据主要是分子结构，数量丰富，而实验测量数据稀缺。因此 ChemBERTa 和 MoLFormer 等化学基础模型在多样分子结构上训练，以便在小规模实验数据集（现实场景中多数少于 100 个数据点）上微调并预测性质。与此同时，GPT-6 Astra 和 Claude Opus 5.5 等通用基础模型几乎在所有公开数据上训练，包括人类关于分子性质及其规律的知识。这里比较化学基础模型与通用基础模型在化学各领域任务上的准确率。引人注目的是，仅有 60 个实验数据点时，通用 GPT-6 Astra 在五个物理化学与量子化学回归任务上优于当前可用的化学基础模型。只有提供 10 倍数据点时，最好的化学基础模型才达到 GPT-6 Astra 的性能：在 1,000 和 10,000 个数据点上，它们分别与 Astra 在 60 和 1,000 个数据点上的性能相当。因此，与直觉相反，前沿通用基础模型比专用化学基础模型更适合指导数据稀缺领域的化学研究，表明学习相关基元（如化学结构）不能替代在相关人类知识上的训练。
关键要点
- 01化学基础模型依赖分子结构预训练，但实验数据稀缺，现实场景常少于 100 个数据点。
- 02仅 60 个数据点时，GPT-6 Astra 在五个物理化学与量子化学回归任务上优于化学基础模型。
- 03化学基础模型需 10 倍数据点（1,000 与 10,000）才分别追平 Astra 在 60 与 1,000 点上的性能。
- 04结论：通用基础模型更适合数据稀缺的化学研究，学习化学结构不能替代人类知识训练。
解读
尚无解读。
原始英文摘要
arXiv:2610.41899v1 Announce Type: cross Abstract: Foundation models (FMs) learn reusable representations from large collections of unlabeled data. In chemistry the corresponding data mainly consists of molecular structures, which are abundant, whereas experimental measurements are scarce. Chemical FMs such as ChemBERTa and MoLFormer were therefore trained on diverse molecular structures to enable fine-tuning on small experimental datasets (less than 100 datapoints in most realistic scenarios) and subsequent property prediction. At the same time, general-purpose FMs such as GPT-6 Astra and Claude Opus 5.5 were trained on virtually all publicly available data, including knowledge of the humankind on molecular properties and laws governing them. Here we compare accuracies of chemical and general-purpose FMs on tasks from various areas of chemistry. Strikingly, general-purpose GPT-6 Astra outperforms the currently eligible chemical FMs on five physicochemical and quantum-chemical regression tasks when only 60 experimental datapoints are available. Only when provided 10x datapoints do the best chemical FMs reach performance of GPT-6 Astra: on 1,000 and 10,000 points they show performance on par with Astra on 60 and 1,000 points, respectively. Thus, counterintuitively, frontier general-purpose FMs are better suited for guiding chemical research in data-scarce areas than specialized chemical FMs, suggesting that learning relevant primitives (e.g. chemical structures) does not substitute training on relevant humankind knowledge.