A3GS: Arbitrary Artistic Style into Arbitrary 3D Gaussian Splatting

摘要

Recently, the field of 3D scene stylization has attracted considerable attention, particularly for applications in the metaverse. A key challenge is rapidly transferring the style of an arbitrary reference image to a 3D scene while faithfully preserving its content structure and spatial layout. Works leveraging implicit representations with gradient-based optimization achieve impressive style transfer results, yet the lengthy processing time per individual style makes rapid switching impractical. In this paper, we propose A^3GS, a novel feed-forward neural network for zero-shot 3DGS stylization that enables transferring any image style to arbitrary 3D scenes in just 10 seconds without the need for per-style optimization. Our work introduces a Graph Convolutional Network (GCN)-based autoencoder aimed at efficient feature aggregation and decoding of spatially structured 3D Gaussian scenes. The encoder converts 3DGS scenes into a latent space. Furthermore, for the latent space, we utilize Adaptive Instance Normalization (AdaIN) to inject features from the target style image into the 3D Gaussian scene. Finally, we constructed a 3DGS dataset using a generative model and proposed a two-stage training strategy for A^3GS. Owing to the feed-forward design, our framework can perform fast style transfer on large-scale 3DGS scenes, which poses a severe challenge to the memory consumption of optimization-based methods. Extensive experiments demonstrate that our approach achieves high-quality, consistent 3D stylization in seconds.