MultiFly:面向标注高效标签迁移与跨模态语义一致性的真实世界多模态航空数据集
MultiFly: A Real-World Multimodal Aerial Dataset with Annotation-Efficient Label Transfer and Cross-Modal Semantic Consistency
Markus Gross · Andreas Greiner · Taehyoung Kim · Sivasubiramaniam Subbiah · Toma\v{z} Coti\v{c} · Sai Bharadwaj Matha · et al.
中文摘要
MultiFly 是一个面向低空无人机(UAV)语义感知的真实世界数据集,涵盖 RGB、热红外、LiDAR 与雷达四种模态。该数据集采集自四个郊区场景,包含 17,272 帧同步样本,提供 15 类语义类别的逐帧标注,并附带标定数据与 GNSS-RTK/IMU 测量信息。为避免昂贵且不一致的逐模态标注,仅对 115 张 RGB 图像进行手工标注,然后通过共享几何表示将标签传播至全部四种模态。该方法为 17,157 张额外 RGB 图像、17,272 张热红外图像、840M LiDAR 点及 3.4M 雷达点生成语义标签。迁移标注与人工标注(留出验证集)的平均一致率为 89.93%,六组模态对的平均语义一致性为 90.94%。进一步在四种模态上建立语义分割基准,揭示了 LiDAR 稠密数据与雷达稀疏数据下不同的架构行为。综上,MultiFly 为多模态航空感知提供了可扩展的基础,据作者所知,是首个为 RGB、热红外、LiDAR 与雷达提供一致逐帧语义标注的公开真实世界低空航空基准。数据见 https://github.com/markus-42/multifly。
关键要点
- 01数据规模:真实世界低空 UAV 数据集,17,272 帧同步样本,15 类语义标注,涵盖 RGB、热红外、LiDAR、雷达四模态,数据量达 840M LiDAR 点与 3.4M 雷达点
- 02标签迁移方法:仅手工标注 115 张 RGB,通过共享几何表示将标签传播至其余 17,157 张 RGB、全部热红外、LiDAR、雷达,迁移标注与人工留出集平均一致率 89.93%
- 03跨模态一致性:六组模态对的平均语义一致性达 90.94%,解决了逐模态标注成本高且不一致的问题
- 04基准结果:在四种模态上建立语义分割基准,发现 LiDAR 稠密数据与雷达稀疏数据上模型架构表现差异显著
- 05贡献定位:首个同时覆盖 RGB、热红外、LiDAR、雷达并具备一致逐帧语义标注的公开真实世界低空航空基准
解读
尚无解读。
原始英文摘要
arXiv:2610.10359v1 Announce Type: new Abstract: We introduce MultiFly, a real-world, low-altitude UAV dataset for semantic perception across RGB, thermal, LiDAR, and radar modalities. MultiFly provides 17,272 synchronized samples from four suburban scenes with frame-wise annotations for 15 semantic classes, together with calibration and GNSS-RTK/IMU measurements. To avoid costly and inconsistent modality-specific annotation, we propagate labels from only 115 manually annotated RGB images through shared geometric representations to all four modalities. This approach generates semantic labels for 17,157 additional RGB images, 17,272 thermal images, 840M LiDAR points, and 3.4M radar points. Transferred annotations achieve 89.93% average agreement with held-out manual annotations, and 90.94% average semantic consistency across all six modality pairs. We further establish semantic segmentation benchmarks for all four modalities, revealing distinct architectural behavior for dense LiDAR and sparse radar data. Taken together, MultiFly provides a scalable foundation for multimodal aerial perception and, to the best of our knowledge, the first public real-world low-altitude aerial benchmark that combines consistent frame-wise semantic annotations for RGB, thermal, LiDAR, and radar. Data at https://github.com/markus-42/multifly.