联邦学习汽车空气动力学代理模型与谱优化器
Federated Learning of Car Aerodynamics Surrogates with Spectral Optimizers
Arman Bolatov · Nikolay Spitsyn · Daniil Medyakov · Vladimir Aletov · Dmitry Bylinkin · Martin Tak\'a\v{c} · et al.
中文摘要
神经代理模型可在数秒内预测汽车气动力,而CFD模拟需数小时,但训练需数千次不同车型的CFD模拟,这些数据被车企视为专有。联邦学习允许多家厂商在不交换数据的情况下训练共享代理模型。研究哪种联邦优化器能使代理模型最接近在汇总数据上训练的模型。在DrivAerNet++上,跨六个客户端训练Transolver代理模型,每个客户端仅持有单一车身类型的车辆,比较三种谱优化器(对每个权重矩阵的更新进行正交化:FedMuon及提出的FedMARS-M和FedOLion)与三种非谱优化器(FedAvg、SCAFFOLD-Adam和FedLion),均采用相同调参协议。FedMuon和FedMARS-M在每轮2至4个本地epoch下,阻力误差与使用Muon的集中式训练相差2%以内。非谱方法在测试的最大本地epoch数16时表现最佳,但阻力误差仍比FedMARS-M高17%至37%。该差距与权重梯度的低稳定秩有关,对此正交化步骤的一阶下降量约为相同谱范数梯度步的两倍。代码:https://github.com/armanbolatov/fedmars_drivaernet
关键要点
- 01问题:训练汽车气动代理模型需大量专有CFD数据,联邦学习可免数据交换但优化器选择不明。
- 02方法:在DrivAerNet++上比较三种谱优化器与三种非谱优化器,跨六个单车身类型客户端训练Transolver。
- 03结果:FedMuon和FedMARS-M阻力误差与集中式训练相差2%以内,非谱方法误差高17%至37%。
- 04机制:权重梯度低稳定秩使正交化步骤一阶下降量约为同谱范数梯度步的两倍。
解读
尚无解读。
原始英文摘要
arXiv:2611.25300v1 Announce Type: new Abstract: Neural surrogates predict the aerodynamic forces on a car in seconds, compared with hours for a CFD simulation, but training them requires thousands of CFD simulations of different car designs, which car manufacturers keep proprietary. Federated learning allows several manufacturers to train a shared surrogate without exchanging data. We study which federated optimizer brings such a surrogate closest to one trained on pooled data. On DrivAerNet++, we train a Transolver surrogate across six clients that each hold cars of a single body type, and we compare three spectral optimizers, which orthogonalize the update of each weight matrix (FedMuon and two methods we propose, FedMARS-M and FedOLion), with three non-spectral ones (FedAvg, SCAFFOLD-Adam and FedLion), all tuned with the same protocol. FedMuon and FedMARS-M reach a drag error within 2% of centralized training with Muon, using two to four local epochs per round. The non-spectral methods perform best with sixteen local epochs, the largest number we tested, and still have a 17 to 37% higher drag error than FedMARS-M. We relate this gap to the low stable rank of the weight gradients, for which an orthogonalized step yields about twice the first-order decrease of a gradient step of the same spectral norm. Code: https://github.com/armanbolatov/fedmars_drivaernet