SAM 3D Body 的人体测量保真度研究
Investigating Anthropometric Fidelity in SAM 3D Body
Aizierjiang Aiersilan · Ruting Cheng · James Hahn
中文摘要
SAM 3D Body 的发布是人体现网格恢复领域的最新进展,在从单张图像生成干净、拓扑一致的网格方面表现出更优性能。借助 Momentum Human Rig (MHR),该模型对遮挡和多样姿态具有鲁棒性。然而,评估揭示了一个具体且一致的局限:模型难以重建细致的人体测量偏差,尤其是在表现出显著形态改变的人群中,如老年肌肉萎缩、脊柱侧弯或妊娠,即使这些特征在输入图像中十分明显。本文将这一现象视为“感知-失真权衡(perception-distortion trade-off)”的副产物,而非模型能力的失败。研究认为,架构上依赖低维参数化 MHR 表示,结合语义不变条件(DINOv3)和基于标注的对齐,共同造成了普遍的“向均值回归”效应。通过分析这些机制,解释个体生物学细节为何被平滑掉。此外,提出未来工作的具体建设性路径,如隐式-显式混合表示和 Medical-in-the-Loop 对齐,以将 SAM 3D Body 的基线性能扩展到高精度医学领域。
关键要点
- 01SAM 3D Body 在单图人体网格恢复中表现优异,但对老年肌肉萎缩、脊柱侧弯、妊娠等形态偏差重建不足。
- 02该局限源于低维参数化 MHR 表示、DINOv3 语义不变条件与标注对齐共同导致的“向均值回归”效应。
- 03作者将问题归因于“感知-失真权衡”,而非模型容量不足。
- 04提出隐式-显式混合表示与 Medical-in-the-Loop 对齐两条路径,以推动模型进入高精度医学领域。
解读
尚无解读。
原始英文摘要
arXiv:2601.06035v2 Announce Type: cross Abstract: The release of SAM 3D Body is a recent development in human mesh recovery, demonstrating improved performance in producing clean, topologically coherent meshes from single images. By leveraging the Momentum Human Rig (MHR), it achieves robustness to occlusion and diverse poses. However, our evaluation reveals a specific and consistent limitation: the model struggles to reconstruct detailed anthropometric deviations, particularly in populations exhibiting distinctive morphological alterations such as geriatric muscle atrophy, scoliosis, or pregnancy, even when these features are prominent in the input image. In this paper, we investigate this phenomenon not as a failure of the model's capacity, but as a byproduct of the "perception-distortion trade-off". We posit that the architectural reliance on the low-dimensional parametric MHR representation, combined with semantic-invariant conditioning (DINOv3) and annotation-based alignment, creates a pervasive "regression to the mean" effect. We analyze these mechanisms to understand why individual biological details are smoothed out. Furthermore, we state our contributions by proposing specific, constructive pathways for future work, such as implicit-explicit hybrid representations and Medical-in-the-Loop alignment, to extend the baseline performance of SAM 3D Body into the high-precision medical domain.