Diff-SemiER: Transparency-Aware Adaptive Fusion Diffusion Model with Generative Prior for Semi-Transparent Eyeglasses Removal

摘要

Existing eyeglasses removal methods primarily focus on opaque or fully transparent lenses. However, when dealing with semi-transparent sunglasses, these methods often corrupt the visible facial details beneath the lenses, thereby degrading the performance of downstream vision tasks. To address this issue, we propose Diff-SemiER, a novel diffusion-based framework for semi-transparent eyeglasses removal that leverages generative priors and transparency-aware adaptive fusion. The proposed framework fully utilizes the visible eye-region information beneath the lenses while retaining sufficient generative flexibility, thereby striking a balance between generation and restoration within semi-transparent regions. Specifically, Diff-SemiER comprises two diffusion branches: the Generative Prior Diffusion Model (GPDM) generates high-quality eyeglass-free facial images via image inpainting, which provides global semantic guidance for highly occluded scenarios. The Transparency-Aware Adaptive Fusion Diffusion Model (TAFDM) employs a Soft Mask-Aware Adaptive Fusion (SMAF) mechanism to adaptively merge generative and restorative features across multiple scales, enabling dynamic trade-offs between generative capability and fine-detail preservation under varying occlusion levels. Furthermore, we design a transmittance-based data synthesis method to construct a large-scale, high-quality dataset of faces with semi-transparent eyeglasses for model training and evaluation. Extensive experimental results demonstrate that Diff-SemiER significantly outperforms state-of-the-art methods in both synthetic and real-world scenarios.