RoboJEPA:机器人潜空间世界模型的规模化
RoboJEPA: Scaling Robotic Latent World Models
Artem Zholus · Nicolas Beltran-Velez · Jianhao Yuan · Sarath Chandar · Tushar Nagarajan · Daniel Severo · et al.
中文摘要
潜空间世界模型(latent world models)在预测未来状态与真实世界规划方面展现出突出能力。然而实践中缺乏一套原则性方法来估计其能力如何随模型规模、数据量与算力扩展,这一开放问题阻碍了领域进展。本工作提出 RoboJEPA,一种基于联合嵌入预测架构(JEPA)的世界模型,在大规模数据集上训练,涵盖 12 种机器人构型(robotic embodiments)。研究表明,RoboJEPA 的想象误差(imagination error),即其潜空间推演(latent rollouts)的误差,遵循关于算力的二阶幂律(second-order power law),从而可在远超出拟合区间的规模上预测模型质量。进一步表明,下游机器人规划性能随算力可预测地提升,且想象误差与其高度相关,使其可作为真实机器人评估的可靠代理指标。最终证明潜空间世界模型可零样本(zero-shot)部署为机器人智能体,仅通过规划单个目标图像,即可在真实硬件上完成需要长视野(long-horizon)规划的任务。全部模型权重以及训练与机器人部署代码均已发布。据作者所知,这是首个针对真实机器人数据训练的多构型机器人世界模型建立扩展定律(scaling law)的工作;RoboJEPA 以 80 亿参数成为迄今训练的最大 JEPA 预测器模型。
关键要点
- 01问题:潜空间世界模型缺乏关于模型规模、数据量与算力的原则性扩展规律,制约机器人领域进展。
- 02方法:基于 JEPA 构建 RoboJEPA,在涵盖 12 种机器人构型的大规模数据集上训练,最大模型达 8B 参数。
- 03结果:想象误差遵循关于算力的二阶幂律,可外推预测更大规模下的模型质量,且与下游规划性能高度相关。
- 04能力验证:RoboJEPA 可零样本部署为机器人智能体,通过单个目标图像完成真实硬件上的长视野规划。
- 05贡献:首个为真实机器人数据训练的多构型世界模型建立扩展定律的工作,并发布全部权重与代码。
解读
尚无解读。
原始英文摘要
arXiv:2610.10515v1 Announce Type: cross Abstract: Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.