FIGFLEX：面向低位宽 LLM 量化协同设计的混合精度 FP-INT GEMM 加速器
FIGFLEX: A Mixed-Precision FP-INT GEMM Accelerator Co-Designed with Low-Bit LLM Quantization
Faraz Tahmasebi · Haocheng Xu · Andrew Trauger · Yomal Perera · Zhenyu Tang · Nael Chowdhury · et al.
中文摘要
仅权重量化使 LLM 激活保持浮点、权重存为低位整数，将主要负载转为浮点-整数(FP-INT)通用矩阵乘法。近期量化器采用混合精度可达到平均 2–3 bit，但硬件进展滞后：GPU 在张量核心乘加前将权重量化解至 FP16；位并行 FP-INT 加速器则使用单一权重精度，低位宽仅节省内存，未减少计算。FIGFLEX 是算法-硬件协同设计的混合精度加速器，其位并行数据通路可随精度降低而扩展吞吐。设计包含：其一，将 FP-INT 乘法重构为移位加法、基数4形式，把乘加融合为精度可调的统一部分积归约，并将部分积约减半；针对 INT3 的新优化利用空闲的基数4叶节点。其二，混合精度量化器依据 GPTQ 无法补偿的误差为各通道组分配位宽，并按输入通道列缩放，以 2 的幂归一化使缩放后激活避开 FP16 次正规范围，使量化模型可直接映射至 FIGFLEX 数据通路，且精度与 GPU 相同。在 LLaMA-2-7B 上，平均 3 bit 和 2 bit 时，WikiText-2 困惑度分别为 5.85 和 8.77，优于未经微调的 GPTQ、AWQ、SliM-LLM、AMQ 和 QuIP#；3-bit 混合精度模型的线性 GEMM 全芯片单位面积吞吐较 AxCore 和 FIGLUT 分别高 1.56 倍和 1.42 倍。
关键要点
- 01现有低位宽量化已降低内存需求，但硬件尚未同步减少计算
- 02移位加法与基数4重构统一乘加归约，约减半部分积
- 03INT3 优化复用空闲基数4叶节点，扩展低位宽数据通路利用率
- 04混合精度量化按通道组分配位宽，并避免激活进入 FP16 次正规范围
- 05LLaMA-2-7B 困惑度与单位面积吞吐均优于多种基线方法
解读
尚无解读。
原始英文摘要
arXiv:2610.30480v1 Announce Type: cross Abstract: Weight-only quantization keeps LLM activations in floating point and stores weights as low-bit integers, turning the dominant workload into floating-point-integer (FP-INT) general matrix multiplication. Recent quantizers reach a 2-3-bit average with mixed precision, but hardware has not kept pace: GPUs dequantize weights to FP16 before the tensor-core multiply-accumulate, and bit-parallel FP-INT accelerators run at a single weight precision, so fewer bits save memory but not computation. In this paper, we present FIGFLEX, an algorithm-hardware co-designed mixed-precision accelerator whose bit-parallel datapath scales throughput as precision falls. FIGFLEX contributes (1) a shift-add, radix-4 reformulation of FP-INT multiplication that fuses the multiply-accumulate into a unified, precision-flexible partial-product reduction and roughly halves the partial products, together with a novel INT3 optimization that reclaims the idle datapath radix-4 leaves at that precision; and (2) a mixed-precision quantizer that assigns each channel group its bit-width according to the error GPTQ cannot compensate, with a per-input-channel column scaling whose power-of-two normalization keeps the scaled activations out of the FP16 subnormal range, so that the quantized model maps directly onto FIGFLEX's datapath with the same accuracy as on a GPU. On LLaMA-2-7B, FIGFLEX attains 5.85 and 8.77 WikiText-2 perplexity at 3- and 2-bit average precision, lower than GPTQ, AWQ, SliM-LLM, AMQ, and QuIP# (without fine-tuning), while delivering 1.56x and 1.42x higher whole-chip throughput per area on the linear GEMMs of a 3-bit mixed-precision model than AxCore and FIGLUT, respectively.