跨星载中断恢复视觉-语言推理
Resuming Vision-Language Inference across Onboard Interruptions
Xiaolong Huang · Yinyuan Zhang · Daliang Xu · Yun Ma · Qing Li · Mengwei Xu · et al.
中文摘要
在近地轨道卫星上运行视觉-语言模型(VLM),可灵活处理观测数据,无需先将数据下传至地面。在商用现货载荷上,单次请求可能跨越多个执行窗口;功耗与热约束、故障触发的重启及载荷资源竞争,都会在推理完成前造成中断。每次关机都会丢弃易失激活值和KV缓存,使未受保护的请求在重启后重复计算。OrbitWeaver通过协同检查点与恢复机制保留VLM推理进度。阶段感知检查点依据视觉编码、语言模型预填充和解码的状态生命周期及计算特征选择恢复边界,在存储流量与重计算之间取得平衡。依赖感知动态恢复可沿层和token分块维度,从部分完成的状态继续执行。系统将检查点和权重读取与重计算重叠,从而缩短重启延迟;恢复规划器可在100 ms内按其调度模型计算最优执行顺序。基于卫星载荷硬件和在轨遥测轨迹的实验显示,相比所评估基线,端到端速度最高提升4.07倍,SSD写入量最多减少3.56倍。在两颗近地轨道卫星上的72小时部署表明,OrbitWeaver能够保留推理进度,并跨总线强制关机的多次中断恢复执行。
关键要点
- 01星载VLM推理易受功耗、热约束、重启和资源竞争影响,关机会丢弃KV缓存并重复计算。
- 02阶段感知检查点按视觉编码、预填充与解码特征选取恢复边界,权衡存储流量和重计算。
- 03依赖感知动态恢复支持跨层与token分块,从部分完成状态恢复并重叠读取与重计算。
- 04恢复规划器在100 ms内计算最优执行顺序,端到端速度最高提升4.07倍。
- 05在轨实验将SSD写入量最多降低3.56倍,72小时部署验证了多次关机后的进度保留与恢复。
解读
尚无解读。
原始英文摘要
arXiv:2610.29183v1 Announce Type: cross Abstract: Executing vision-language models (VLMs) aboard low-Earth orbit satellites enables flexible processing of observation data without first downlinking it to the ground. On commercial off-the-shelf payloads, however, a request may span multiple execution windows: power and thermal constraints, fault-triggered reboots, and competing payload demands can interrupt inference before it completes. Each shutdown discards volatile activations and KV caches, forcing an unprotected request to repeat work after restart. We present OrbitWeaver, a framework that preserves VLM inference progress across onboard interruptions through coordinated checkpointing and recovery. Its phase-aware checkpointing selects recovery boundaries according to the state lifetimes and computational characteristics of vision encoding, language-model prefill, and decoding, balancing storage traffic against recomputation. Its dependency-aware dynamic recovery schedules restoration from partially completed states across the layer and token-chunk dimensions. By overlapping checkpoint and weight reads with recomputation, it reduces restart latency; the recovery planner computes an optimal execution order under its scheduling model within 100 ms. Experiments on satellite payload hardware using on-orbit telemetry traces show up to a 4.07-fold end-to-end speedup and a 3.56-fold reduction in SSD write volume over the evaluated baselines. A 72-hour deployment on two low-Earth orbit satellites demonstrates that OrbitWeaver retains inference progress and resumes execution across repeated bus-enforced shutdowns.
同方向论文 · cs.DC
查看全部 →复现可变部分:面向域随机化具身仿真的资源共享
2610.33582KnitTrans:面向高效大语言模型强化学习的协同轨迹传输
2610.32389边缘计算赋能 LLM 训练普惠化:挑战、系统与未来路线
2610.32215InferSlice:面向电信广域网 SLO 驱动 LLM 推理的参考架构
2610.32638GPU 加速构建 Gabriel 图:并行化策略及其在大间隔分类与图聚类中的应用
2610.28759Difflet: 基于 AWS Trainium 的 Diffusion Transformer 快速推理引擎
2610.23047