cs.DC提交于 2026-10-07已译

T-CCL：基于 Tensor Memory Accelerator 的资源高效高性能集合通信库

T-CCL: Resource Efficient and Performant Collective Communication using Tensor Memory Accelerator

Keyvan Dadashzadeh · Yuehong Zhou · Minyu Cui · Miquel Pericas

中文摘要

大规模基于 Transformer 的模型越来越多地依赖多 GPU 执行，需要 GPU 之间频繁进行集合通信。现有通信库通常依赖许多 GPU 线程以获得高带宽或低延迟，导致在流式多处理器（SM）一侧的资源占用很大。这种占用会限制其他 GPU 工作可用的资源，特别是在通信与计算并发执行时。因此，高效的集合通信不仅要实现高集合性能，还应降低其 SM 侧资源使用。本文提出 T-CCL，一种基于 Tensor Memory Accelerator（TMA）的资源高效集合通信库，支持节点内通信。T-CCL 将数据搬运与归约操作均卸载到 TMA，将每次集合操作作为一组流水线化的异步 TMA 操作执行，在保持高带宽的同时减少了集合通信所需的 SM 资源。在 AllReduce、AllGather 和 ReduceScatter 三种集合操作上评测，T-CCL 在通信资源不受限时最高达到 NCCL 的 2.4 倍，在受限资源预算下最高达到 3.42 倍，与 NCCL 最新的对称内存（symmetric-memory）内核相比仍具竞争力，且在所分析的案例中占用相同乃至更少 SM。在 GEMM 与集合通信重叠的案例研究中，将通信后端从 NCCL 切换到 T-CC 后，相较于串行基线的算子级平均加速比在双 GPU 上从 1.12 倍提升到 1.25 倍，在四 GPU 上从 1.04 倍提升到 1.14 倍，原因是 T-CCL 用更少 SM 完成通信，为其留出更多 SM 供重叠的 GEMM 使用。集成到 vLLM 作为通信后端时，T-CCL 在对话类与解码密集型负载上各批次大小下，端到端推理吞吐量较 vLLM 自动后端调度最高提升 1.31 倍。

关键要点

01问题：多 GPU 大模型推理中集合通信占用大量 SM 资源，限制可与计算并发的资源量，单纯追求带宽或延迟不够。
02方法：T-CCL 将数据搬运与归约操作全部卸载到 Tensor Memory Accelerator（TMA），以流水线化异步 TMA 操作实现集合通信，显著降低 SM 占用。
03结果：在 AllReduce、AllGather、ReduceScatter 上对标 NCCL，无资源限制时最高 2.4 倍，受限资源下最高 3.42 倍，且占用 SM 数相同或更少。
04结果：GEMM–集合通信重叠场景下算子级平均加速比从 1.12 倍提升至 1.25 倍（双 GPU）、1.04 倍提升至 1.14 倍（四 GPU）。
05集成：作为 vLLM 通信后端时，端到端推理吞吐量较其自动后端调度最高提升 1.31 倍，并在所有评测批次大小下均更优。

解读

尚无解读。

原始英文摘要

arXiv:2610.07098v1 Announce Type: cross Abstract: Large transformer-based models increasingly depend on multi-GPU execution, which requires frequent collective communication among GPUs. Existing communication libraries often rely on many GPU threads to achieve high bandwidth or low latency, resulting in a large streaming multiprocessor (SM)-side resource footprint. This footprint can limit the resources available to other GPU work, particularly when communication and computation execute concurrently. Thus, efficient collective communication should not only achieve high collective performance but also reduce its SM-side resource usage. This paper presents T-CCL, a resource-efficient collective communication library based on the Tensor Memory Accelerator (TMA) for intra-node communication. T-CCL offloads both data movement and reduction operations to TMA and executes each collective as a pipelined series of asynchronous TMA operations, reducing the SM resources required for collective communication while maintaining high bandwidth. Evaluated across AllReduce, AllGather, and ReduceScatter collectives, T-CCL outperforms NCCL by up to 2.4x with unrestricted communication resources and up to 3.42x under restricted resource budgets, remains competitive with NCCL's recent symmetric-memory kernels, and occupies the same or fewer SMs in profiled cases. In a GEMM-collective overlap case study, switching the communication backend from NCCL to T-CCL raises the average operator-level speedup over a sequential baseline from 1.12x to 1.25x on two GPUs and from 1.04x to 1.14x on four GPUs, as T-CCL uses fewer SMs for communication, leaving more SMs available to the overlapped GEMM. Integrated into vLLM as a communication backend, T-CCL improves end-to-end inference throughput over vLLM's automatic backend dispatch by up to 1.31x, outperforming it at every evaluated batch size on both the conversation and decode-heavy workloads.