跳到正文
返回论文列表
cs.CV提交于 已译

WAMJET:世界动作模型加速工具链

WAMJET: A Harness for World Action Model Acceleration

Le Chen · Lixin Liu · Jan Schneider · Zeju Qiu · Simon Guist · Bernhard Sch\"olkopf · et al.

中文摘要

世界动作模型(WAM)利用预训练视频基础模型完成机器人操控任务,但其庞大的骨干网络与视频-动作联合预测机制带来高昂推理成本。现有加速技术虽能降低成本,却需针对每个模型与硬件平台投入大量工程化工作。为突破该瓶颈,本文提出 WAMJET——一种智能体驱动(agentic)的工具链,通过为编码智能体配备可复用的优化指引、测量与验证工具来加速 WAM 推理。WAMJET 采用瓶颈驱动的工作流:智能体先对推理过程进行性能剖析(profile),再针对性修改代码、验证效果,并在瓶颈动态变化时迭代优化加速栈,同时保持动作质量。实验覆盖 6 个 WAM、3 个编码智能体与 2 种 GPU 架构。WAMJET 相对上游实现取得最高 9.95 倍无损加速;近似优化与硬件感知优化带来进一步时延降低,且成功率相当。结果表明 WAMJET 可为 WAM 部署生成有效的加速方案。

关键要点

  1. 01问题:WAM 推理成本高昂,现有加速技术需针对每个模型与硬件平台大量工程化
  2. 02方法:WAMJET 作为智能体工具链,提供可复用优化指引与测量验证工具,按瓶颈驱动流程迭代重构
  3. 03实验规模:覆盖 6 个 WAM、3 个编码智能体、2 种 GPU 架构
  4. 04结果:相对上游实现最高 9.95 倍无损加速;近似与硬件感知优化进一步降低时延,成功率相当
  5. 05价值:降低 WAM 部署门槛,可自动产出有效的加速方案

解读

尚无解读。

原始英文摘要

arXiv:2610.03797v2 Announce Type: replace-cross Abstract: World Action Models (WAMs) leverage pretrained video foundation models for robot manipulation, but their large backbones and video-action co-prediction are expensive. Although existing acceleration techniques offer many ways to reduce this cost, selecting and composing them requires substantial engineering for each model and hardware platform. To tackle this bottleneck, we present WAMJET, an agentic harness that accelerates WAM inference by equipping coding agents with reusable optimization guidance and measurement and validation tools. WAMJET follows a bottleneck-driven workflow where the agent profiles inference, modifies targeted code, validates effects, and iteratively refines the acceleration stack as bottlenecks shift, while preserving action quality. Experiments span six WAMs, three coding agents, and two GPU architectures. WAMJET achieves up to 9.95x lossless speedup over upstream implementations. Approximation and hardware-aware optimization yield additional latency reductions, with comparable success rates. The results show that WAMJET can produce effective acceleration stacks for WAM deployment.

同方向论文 · cs.CV

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 3

视频github.com/f/awesome-chatgpt-prompts·外部收录·

Claude prompt: 10-second app ad for Reels and TikTok (9:16)

Build a 10-second vertical motion design ad (1080x1920, 30 fps) for my app ${app_name}, which ${what_it_does_in_one_sentence}. Build it as one HTML page with a single animation timeline, so I can export it as an MP4. Look: very clean and f…

视频github.com/f/awesome-chatgpt-prompts·外部收录·

Claude prompt: 12-second SaaS product film in 16:9

Build a 12-second motion design product film in 16:9 (1920x1080, 30 fps) for my software product ${name}, which ${what_it_does_in_one_sentence}. Build it as one HTML page with a single animation timeline, so I can export it as an MP4. Look…