CityRAG：通过空间锚定的视频生成步入城市
CityRAG: Stepping Into a City via Spatially-Grounded Video Generation
Gene Chou · Charles Herrmann · Kyle Genova · Boyang Deng · Songyou Peng · Bharath Hariharan · et al.
中文摘要
针对生成具有空间锚定、3D 一致且可导航环境（即真实地点的模拟）的问题，现有视频生成模型能生成与文本（T2V）或图像（I2V）提示一致的合理序列，但下游应用（如自动驾驶和机器人仿真）需要能在任意天气条件和动态物体配置下重建真实世界。为此，提出 CityRAG，一种视频生成模型，利用大规模地理配准数据作为上下文，将生成锚定到物理场景，同时保留复杂运动和外观变化的学习先验。CityRAG 依赖时间上未对齐的训练数据，使模型学会从瞬态属性中语义解耦底层场景。实验表明，CityRAG 能生成连贯的、长达数分钟且物理锚定的视频序列，在数千帧上保持天气和光照条件，实现回环闭合，并导航复杂轨迹以重建真实世界地理。
关键要点
- 01问题：现有视频生成模型无法在任意天气和动态物体下重建真实地点，难以支撑自动驾驶与机器人仿真。
- 02方法：CityRAG 以大规模地理配准数据为上下文锚定物理场景，并用时间未对齐数据解耦场景与瞬态属性。
- 03结果：可生成数分钟连贯物理锚定视频，数千帧保持天气光照，实现回环闭合与复杂轨迹导航。
- 04局限：摘要未提及具体局限。
解读
尚无解读。
原始英文摘要
arXiv:2604.19741v3 Announce Type: replace Abstract: We address the problem of generating a 3D-consistent, navigable environment that is spatially grounded: a simulation of a real location. Existing video generative models can produce a plausible sequence that is consistent with a text (T2V) or image (I2V) prompt. However, the capability to reconstruct the real world under arbitrary weather conditions and dynamic object configurations is essential for downstream applications including autonomous driving and robotics simulation. To this end, we present CityRAG, a video generative model that leverages large corpora of geo-registered data as context to ground generation to the physical scene, while maintaining learned priors for complex motion and appearance changes. CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes. Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.