平衡层次表示与量化：文本嵌入中的性能权衡
Balancing hierarchical representation and quantization: A performance trade-off in text embeddings
Kai Ponel · Marco Polignano · Erasmo Purificato · Aylin Julia Temizkan · Marco Levantesi · Giovanni Semeraro · et al.
中文摘要
层次化文本嵌入模型提供可变长度表示，量化则降低数值精度；两者都旨在提升计算效率。本工作是一项实证基准研究，考察这两种正交且已成熟的技术如何相互作用。使用四个基于 transformer 的嵌入模型，在 Massive Text Embedding Benchmark (MTEB) 的 38 个任务上评估，覆盖 1D 和 2D 层次表示与 32 位、8 位及二值量化的组合，每个任务共 168 种配置。分别报告每个效率维度：理论每向量存储和端到端评估时间。在 Friedman 检验与 Nemenyi 事后分析下，int8 量化在配对分类和重排序上与 float32 统计不可区分，整体最多损失 1.5 个百分点，而二值量化在每个任务类别上均显著更差。嵌入维度减半一次在 m=16 以上统计不可检测；连续两次减半则总是可检测。将 8 倍降维与 int8 结合，每向量存储减少 32 倍，达到基线效果的 94.4%。由于评估时间仅测量一次，我们将其视为指示性：未发现量化加速评估的证据，而从中间层采样可显著缩短评估时间。检索性能的下降远快于其他所有类别，因此按任务类别报告发现。
关键要点
- 01层次表示与量化两种效率技术正交，在 MTEB 38 个任务上系统评估其交互影响。
- 02int8 量化与 float32 统计不可区分，整体最多损失 1.5 个百分点；二值量化在所有类别显著更差。
- 038 倍降维加 int8 实现每向量存储减少 32 倍，保留基线效果的 94.4%。
- 04未发现量化加速评估的证据，中间层采样可显著缩短评估时间；检索性能下降最快。
解读
尚无解读。
原始英文摘要
arXiv:2611.06859v1 Announce Type: new Abstract: Hierarchical text embedding models offer variable-length representations, while quantization reduces numerical precision; both aim to improve computational efficiency. This work is an empirical benchmark study of how these two orthogonal, already-established techniques interact. We evaluate 38 tasks from the Massive Text Embedding Benchmark (MTEB) using four transformer-based embedding models, covering 1D and 2D hierarchical representations combined with 32-bit, 8-bit, and binary quantization, for 168 configurations per task. We report each efficiency dimension separately: theoretical per-vector storage and end-to-end evaluation time. Under a Friedman test with Nemenyi post-hoc analysis, int8 quantization is statistically indistinguishable from float32 for pair classification and re-ranking and costs at most 1.5 percentage points overall, whereas binary quantization is significantly worse across every task category. Halving the embedding dimension once is not statistically detectable above $m=16$; two successive halvings always are. Combining an 8x dimensionality reduction with int8 yields a 32x reduction in per-vector storage at 94.4% of baseline effectiveness. Because evaluation times were measured once, we treat them as indicative: we find no evidence that quantization speeds up evaluation, whereas sampling from intermediate layers shortens it substantially. Retrieval degrades far faster than every other category, so we report findings per task category.