EDGAR:通过合成失败样本的高效扩散驱动高斯头像精修
EDGAR: Efficient Diffusion-Driven Gaussian Avatar Refinement via Synthetic Failure
Kwan Yun · Minhyeok Bang · Colette Suhjung Koo · Junyong Noh
中文摘要
近期高斯泼溅(Gaussian splatting)的进展使高效、逼真的头部头像重建成为可能,但基于单目视频或图像的方法仍受几何与纹理约束不足的困扰,产生模糊、拉伸和时间不稳定等伪影。EDGAR(Efficient Diffusion-Driven Gaussian Avatar Refinement)是一个条件扩散框架,通过修正伪影并保持身份保真度来提升高斯头像质量。为训练 EDGAR,构建了 EDGAR-SET,一个由真实采集与合成渲染样本组成的混合三元组数据集,通过将退化渲染与真值及参考图像配对来模拟真实的高斯头像失败案例。EDGAR 通过融合模块精修退化帧,将自监督 Sapiens 模型提取的身份特征注入一步扩散骨干,在不牺牲效率的前提下有效去除伪影。模型采用混合专家(Mixture-of-Experts, MoE)架构以应对多样的退化类型与严重程度。精修后的多视角渲染随后作为伪目标优化单一底层高斯头像,使最终头像无需对每帧输出应用 EDGAR 即可渲染。在三种高斯头像重建方法、各三种设置下的实验显示,感知与重建指标均优于原始输出及 SDEdit 风格的精修基线。跨视角评估表明校准一致性提升,大偏航角与张嘴表情测试验证了极端设置下的鲁棒性。结果共同表明存在一个可恢复外观伪影的有效工作区间。代码见 https://kwanyun.github.io/edgar/
关键要点
- 01单目视频或图像驱动的高斯头像重建存在几何纹理约束不足,导致模糊、拉伸与时间不稳定伪影。
- 02构建 EDGAR-SET 混合三元组数据集,用退化渲染与真值、参考图像配对模拟真实失败案例。
- 03融合 Sapiens 身份特征与一步扩散骨干,并用 MoE 应对多样退化类型与严重程度。
- 04精修渲染作伪目标优化单一高斯头像,最终渲染无需逐帧运行 EDGAR。
- 05三种重建方法各三种设置下感知与重建指标均优于原始输出及 SDEdit 基线。
解读
尚无解读。
原始英文摘要
arXiv:2611.07909v1 Announce Type: cross Abstract: Recent advances in Gaussian splatting have enabled efficient and photorealistic head avatar reconstruction, yet monocular video-based or image-based methods still suffer from under-constrained geometry and texture, leading to artifacts such as blurring, stretching, and temporal instability. We present EDGAR (Efficient Diffusion-Driven Gaussian Avatar Refinement), a conditional diffusion framework designed to enhance the quality of Gaussian avatars by correcting artifacts while preserving identity fidelity. To train EDGAR, we construct EDGAR-SET, a hybrid triplet dataset composed of real-captured and synthetic-rendered samples, which simulates realistic Gaussian avatar failure cases by pairing degraded renderings with ground-truth and reference images. EDGAR refines degraded frames through a fusion block that injects identity features extracted by the self-supervised Sapiens model into a one-step diffusion backbone, enabling effective artifact removal without sacrificing efficiency. Our EDGAR model incorporates a Mixture-of-Experts (MoE) architecture to address diverse degradation types and severity levels. The refined multi-view renderings are subsequently used as pseudo-targets to optimize a single underlying Gaussian avatar, so that the final avatar can be rendered without applying EDGAR to every output frame. Experiments on three different Gaussian avatar reconstruction methods, each evaluated under three different settings, show improvements in both perceptual and reconstruction measures over the raw outputs and an SDEdit-style refinement baseline. In addition, cross-view evaluation demonstrates improved calibrated consistency, while tests under large yaw and open-mouth expressions verify robustness under extreme settings. These results collectively indicate a useful operating regime for recoverable appearance artifacts. The code is available at https://kwanyun.github.io/edgar/