cs.AR提交于 2026-10-02已译

ShatterQuant: 打破均匀精度——基于块级混合精度的脉动Transformer硬件加速器

ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator

Mikolaj Walczak · Edward Humes · Chao Fang · Marian Verhelst · Tinoosh Mohsenin

中文摘要

由于传统加速器对张量内异构精度的支持有限，神经网络量化仍然主要局限于按张量分配精度。提出ShatterQuant，一个软硬件协同设计框架，通过为权重投影的块分配独立位宽来实现每个张量内的混合精度量化。ShatterQuant将精度粒度与PE配置耦合，使得每个精度决定有效的块高度。提出(1)一种硬件感知的训练后量化方法，基于块级标准差和权重敏感度为张量内分配精度；(2)支持1/2/4/8-bit权重精度、精度依赖的PE配置、块重缩放以及集成softmax和分段线性非线性的ShatterQuant Transformer加速器；以及(3)使用TSMC 16nm PDK在1GHz下实现的模型-硬件权衡评估，达到1.5 TOPS、760 GOPS/mm²面积效率和2.8 TOPS/W能源效率。在DeiT和ImageNet-1K上，ShatterQuant实现与最先进的混合精度技术相比精度差距在3.3%以内的同时，使用了2位更低的有效位宽，而在PixelDiT上展示了相当的生成质量。ShatterQuant展示了如何通过软硬件协同实现细粒度张量内混合精度。

关键要点

01问题：传统硬件加速器缺乏张量内异构精度支持，神经网络量化局限于按张量统一分配精度
02方法：软硬件协同设计框架，为权重投影的各块独立分配位宽，精度粒度与PE配置耦合
03硬件：ShatterQuant Transformer加速器支持1/2/4/8-bit权重精度，含精度依赖PE配置、块重缩放及集成非线性函数
04性能：TSMC 16nm工艺1GHz下实现1.5 TOPS吞吐量、760 GOPS/mm²面积效率、2.8 TOPS/W能效
05结果：DeiT/ImageNet-1K上精度损失<3.3%且有效位宽降低2位；PixelDiT生成质量与全精度相当

解读

尚无解读。

原始英文摘要

arXiv:2610.00207v1 Announce Type: cross Abstract: Due to limited support for intra-tensor heterogeneous precision in conventional accelerators, neural network quantization remains largely restricted to per-tensor precision assignment. We present ShatterQuant, a hardware-software co-designed framework enabling mixed-precision quantization within each tensor by assigning independent bit-widths to blocks of a weight projection. ShatterQuant couples precision granularity with PE configuration, such that each precision determines an effective block height. We introduce (1) a hardware-aware post-training method that assigns intra-tensor precision based on block-level standard deviation and weight sensitivity; (2) the ShatterQuant Transformer Accelerator supporting 1/2/4/8-bit weight precision, precision-dependent PE configuration, block rescaling, and integrated softmax and piecewise-linear nonlinearities; and (3) an evaluation of model-hardware tradeoffs using an implementation in the TSMC 16nm PDK operating at 1 GHz, achieving 1.5 TOPS, 760 GOPS/$mm^2$ area efficiency, and 2.8 TOPS/W energy efficiency. On DeiT and ImageNet-1K, ShatterQuant achieves accuracy within $3.3\%$ of state-of-the-art mixed-precision techniques while using a 2 bit lower effective bitwidth, while for PixelDiT demonstrates comparable generation quality. ShatterQuant demonstrates how fine-grained intra-tensor mixed-precision can be realized through hardware-software co-design.