Freq-RWKV: Granularity-Aware Spatial-Frequency Synergy via Dual-Domain Recurrent Scanning for Pan-sharpening

摘要

Pan-sharpening aims to improve the spatial resolution of low-resolution multispectral (LRMS) image by integrating high-frequency information from corresponding texture-rich panchromatic (PAN) image. While RWKV architecture has demonstrated remarkable global perception with linear computational efficiency in vision tasks, its inherent sequential scanning mechanism critically compromises local spatial coherence, hindering high-frequency reconstruction. To bridge this gap, we tailor Freq-RWKV, the first spatial-frequency adaptive RWKV featuring dual-domain scanning where wavelet-guided path selection dynamically modulates scanning granularity and orientation according to spectral-spatial information density distributions. Building upon this innovation, we architect the hierarchical U-shaped fusion network that strategically coordinates granularity-aware scanning across spatial and frequency domains, enabling adaptive trade-offs between performance and complexity. The U-shaped architecture implements coarse-to-fine enhancement: in the encoding stage, Coarse Structural Interaction (CSI-RWKV) module preserves geometric dependencies via window-constrained recurrent scanning while encoding structural priors into LRMS features; during decoding, the Fine-grained Frequency Interaction (FFI-RWKV) module performs edge-aware refinement through differentiable frequency-adaptive window partitioning, where multi-scale spectral wavelet attention prioritizes high-frequency PAN components extracted via discrete wavelet transform (DWT). This hybrid decomposition strategy maintains spectral integrity through approximation coefficients while detail coefficients regulate frequency-gated fusion thresholds. Extensive experiments on multiple satellite datasets validate the effectiveness of the proposed method.