Modeling and Identifying Distractors with Curriculum for Robust 3D Gaussian Splatting

摘要

3D Gaussian Splatting (3DGS) is a recent popular technique that can reconstruct the radiance field representation of the scene efficiently. However, the naive 3DGS algorithm is easily affected by noisy pixels from transient and dynamic objects. To resolve this matter and enable the robust learning of 3DGS, previous work proposed to generate a binary mask from per-pixel training loss or an image segmentation result. However, such modeling of the distractors is not adaptive to the 3DGS model learning process, and might lead to wrong identification of noisy pixels and would affect the reconstruction performance. Instead, we propose to learn a soft mask of the likelihood of the distractors. Moreover, we develop techniques to model the spatial pattern of distractors and learn them from a design curriculum, avoiding confusion between clean and noisy pixels. Our method demonstrates state-of-the-art performance on robust novel view synthesis from distractor images, evaluated on major benchmarks of this task.