Cycle-Consistent Mamba-Based Registration-Fusion Joint Network for Unregistered Hyperspectral Image Super-Resolution

摘要

Hyperspectral image super-resolution (HSI-SR) has attracted significant attention in high-resolution HSI reconstruction. Most existing fusion-based HSI-SR methods assume that multi-source images are perfectly registered, which is impractical due to varying imaging conditions. Furthermore, methods that consider the registration issue typically treat registration and fusion as two separate steps, resulting in the accumulation of registration errors during the fusion process. To address these issues, we propose a Cycle-Consistent Mamba-Based Registration-Fusion Joint Network (CCM-RFJN), which step-wise optimizes the Registration-Fusion Unified Module (RFUM) through multiple cyclic iterative SR processes. Specifically, in each SR iteration, we map the super-resolved HR-HSI obtained through the RFUM back to the unregistered LR-HSI for the next SR, with cycle-consistency constraints imposed on both LR-HSI and HR-HSI to adaptively optimize the RFUM based on the reciprocal training strategy. In RFUM, we integrate the proposed Interactive Mamba Registration Module (IMR) and Dual-attention Mamba Fusion Module (DAMF), thereby achieving registration-fusion joint optimization. Specifically, IMR is developed to incorporate the interactive Mamba encoder into a pyramid architecture to facilitate multi-level information interactions, generating the deformation field to correct non-rigid misalignments. DAMF is designed to utilize the dual-attention Mamba mechanism to highlight and aggregate key features, thereby enhancing fusion performance. Experiments on three public datasets demonstrate that CCM-RFJN achieves the state-of-the-art performance. The code is available at https://github.com/Jiahuiqu/CCM-RFJN.