Breaking the Spatial-Temporal Consistency Constraint: Towards Reference-Based Hyperspectral Image Super-Resolution

摘要

Currently, fusion-based hyperspectral image super-resolution (fusion-based HSI-SR) has become an efficient technology to improve the spatial resolution of hyperspectral images. However, in real scenarios, it may not be possible to obtain high-resolution multispectral images (HR-MSI) of the same temporal and region corresponding to low-resolution hyperspectral images (LR-HSI) due to the limitations of imaging conditions and environmental changes. In view of this spatial-temporal constraint, it becomes a feasible solution to regard HR-MSI, which has similar spatial structure and semantics to LR-HSI, as a reference to assist in reconstruction. Therefore, this paper proposes a Cross-Correlation & Self-Similarity Guided Texture Transfer Network (C2S2TNet), which utilizes the texture details of HR-MSI and the self-similarity information of LR-HSI to achieve reference-based hyperspectral image super-resolution. Specifically, we design a Cross-Correlation & Self-Similarity Guided Cluster-Aware Matching (C2S2CAM) strategy, which realizes multi-correspondence texture matching and feature aggregation in non-local regions based on dynamic clustering and cluster-aware graph structure, effectively alleviating the misuse and underuse of information. In addition, we also propose a Spectral-Spatial State-Space Fusion Module (S2-SSFM) based on the state-space model to perform feature fusion and enhancement in both spatial and spectral domains to ensure that the target HR-HSI maintains the spatial-spectral structural consistency with the LR-HSI. Experimental verification shows that C2S2TNet can achieve excellent performance in cross-temporal and cross-regional scenarios, confirming the effectiveness of this method. Code can be accessed at https://github.com/Jiahuiqu/C2S2TNet.