Deep Multi-Level Contrastive Clustering for Multi-Modal Remote Sensing Images

摘要

Multi-modal remote sensing image clustering aims to group similar pixels into the same cluster and separate dissimilar ones by leveraging the consistency and complementary information across multiple modalities, without relying on label guidance. Most existing deep learning-based methods address this task through a two-stage pipeline of feature learning followed by clustering, or adopt simple instance-level contrastive learning frameworks. In this paper, we propose an end-to-end deep multi-level contrastive clustering (DMLCC) model for multi-modal remote sensing images. The proposed DMLCC consists of three key components. Specifically, modality-specific vision encoders are initially employed to extract preliminary feature representations tailored to the characteristics of each modality. Spatial-spectral cross-modal fusion is then performed by integrating dedicated spatial and spectral feature extractors alongside a cross-modal fusion block to effectively capture and align complementary information across modalities. Finally, multi-level contrastive learning is applied to enhance feature discriminability through instance-level contrastive learning, while simultaneously promoting cluster separability via cluster-level contrastive learning. The network is trained in an end-to-end manner by integrating the three components to directly yield the clustering results. Extensive experiments on three datasets demonstrate that the proposed DMLCC outperforms state-of-the-art methods. Our code is publicly available at https://github.com/ZhangYongshan/DMLCC.