VaF-LangSplat: Voxel-Aware Fusion Language Gaussian Splatting

摘要

Efficient and precise open-vocabulary 3D scene segmentation remains a critical challenge in computer vision. While current leading methods encode CLIP language features into 3D Gaussians to achieve high segmentation accuracy and fast inference speeds, they suffer from point ambiguity issues caused by separately training on multi-level 2D semantic masks. This approach not only compromises time and space efficiency but also degrades accuracy when selecting optimal semantic levels. To overcome these limitations, we propose Voxel-Aware Fusion Language Gaussian Splatting (VaF-LangSplat), a novel framework that jointly optimizes geometric and semantic representations. Our approach first voxelizes 3D Gaussians using sparse point clouds and lightweight MLP decoders, effectively disentangling language features from geometric attributes. This enables simultaneous training across arbitrary semantic levels with minimal overhead. Crucially, we introduce Fusion Language Splatting, which aligns geometric and multi-level semantic distributions to sharpen boundary definitions while eliminating redundant Gaussian expansions. The voxel-aware representation further enhances robustness against motion blur and lighting variations. Experiments on open-vocabulary 3D localization and segmentation tasks demonstrate that VaF-LangSplat outperforms LangSplat (the prior state-of-the-art) with significant improvements in both segmentation/localization accuracy and efficiency: 4X faster training and 15X reduced storage requirements.