Wild3A: Novel View Synthesis from Any Dynamic Images in Seconds

摘要

The core issue in novel view synthesis lies in how to handle dynamic scenes that are common in the real world, such as dynamic objects, occlusions, and varying luminance. Current 3D Gaussian Splatting-based methods perform excellently in static scenes but often rely on fixed camera parameters, precise semantic prior segmentation, or specially designed rendering loss functions when dealing with dynamic scenes. These additional pieces of information limit the method's generalization ability, real-time performance, and application in AR, VR, and multimedia. To address these issues, we propose Wild3A, a comprehensive end-to-end integrated framework: it directly regresses 3D point positions and initial point confidence via the MASt3R's Transformer and integrates a Bayesian estimation module based on multimodal information fusion. We adopt a self-supervised joint optimization approach for scene representation and camera parameters. Extensive experiments on both public and private datasets show that Wild3A effectively eliminates visual artifacts in various dynamic scenes, achieves state-of-the-art results across multiple tasks, and achieves real-time rendering at 1000+ FPS.