Mesh-Native 词元化:将基础模型适配到非结构化数据
Mesh-Native Tokenisation: Retrofitting Foundation Models to Unstructured Data
Georg Maierhofer · Miles Cranmer · Payel Mukhopadhyay
中文摘要
物理基础模型是构建时空系统通用仿真器的一种日益流行的方法,它们需要词元化层,将物理场数据提升到适合动态处理的潜在表示。该层通常选用卷积编码器,以局部感知方式提取场信息,训练效率高。但这一选择无法应用于以非结构化网格表示的数据(例如在曲面边界域上的仿真),这一大类科学计算问题无法用卷积编码器处理。这类数据通常需要图神经网络架构,其训练成本更高,并且现有形式与 CNN 对应的网络编码方式不同,限制了物理仿真器在这些数据形态间的可迁移性。本文引入 Mesh-Native 卷积(MNC),一种局部感知的词元化层,以类比 CNN 的方式对基于非结构化网格的数据进行编码。该方法生成的模型可在网格数据上一次性训练,然后部署到任意非结构化网格,即使是规则网格局部微调极少。实验表明,该词元化方法可将预训练于网格数据的基础模型高效适配到非结构化网格,并给出理论结果证明其与经典 CNN 编码器的一致性。结果显示,这种适配方法在零样本部署下多情形即可达到可用精度,经少量 epoch 微调后即超越现有基线,为两种数据形态间架起高效桥梁。
关键要点
- 01问题:现有物理基础模型依赖卷积编码器进行词元化,难以处理曲面边界等场景下的非结构化网格数据,而图神经网络训练成本高且编码方式与 CNN 不兼容。
- 02方法:提出 Mesh-Native Convolution(MNC),一种局部感知的词元化层,以类比 CNN 的方式对非结构化网格数据编码,并理论证明其与经典 CNN 编码器的一致性。
- 03结果:模型可在网格数据上一次性训练后部署到任意非结构化网格,零样本部署多情形即达可用精度,少量 epoch 微调后超越现有基线。
- 04价值:为网格数据与非结构化网格数据两种形态间的物理仿真器迁移搭建高效桥梁,使预训练基础模型可被适配到非结构化数据场景。
解读
尚无解读。
原始英文摘要
arXiv:2610.12618v1 Announce Type: cross Abstract: Physics foundation models are an increasingly popular approach for building versatile emulators for spatiotemporal systems. They require tokenisation layers that lift physical field data into a latent representation suitable for dynamic processing. A typical choice for this layer is a convolutional encoder, which lifts field information in a locality-aware way and can be trained very efficiently. However, this choice cannot be applied to a large class of scientific computing problems where data are represented on unstructured meshes, for example simulations on domains with curved boundaries. Such data typically require graph neural network architectures, which are more expensive to train and, in the form currently used, encode information differently from their CNN counterparts, thus limiting the portability of physics emulators between these data regimes. In this work we introduce Mesh-Native Convolution (MNC), a locality-aware tokenisation layer that encodes unstructured mesh-based data in a way that is analogous to CNNs. This approach leads to models that can be trained once on mesh data and then deployed to arbitrary unstructured meshes, even if these are irregular, with minimal retraining. We show that this tokenisation approach can be used to retrofit grid-pretrained foundation models efficiently on unstructured meshes and provide theoretical results to confirm the consistency of this approach with classical CNN encoders. We demonstrate the favourable performance of this retrofitting approach, which in many cases reaches acceptable accuracy even in zero-shot deployment, and outperforms existing baselines after only a few epochs of fine-tuning, thus providing an efficient bridge between the two data regimes.