Exploring Global Correlations via Polarity Memory for Multispectral Demosaicing

摘要

Multispectral image demosaicing aims to reconstruct full band multispectral images from a compressed spectral mosaic images. Although existing learning-based methods have made progress in multispectral image demosaicing, there still exist intrinsic performance bottlenecks due to the heavy undersampling according to mosaic pattern. To address this issue, we propose Polarity memory network with quant attention to establish global correlation, thus reconstructing high-quality multispectral images from compressed spectral mosaic images. Our proposed Polarity memory network adaptively encapsulates reconstruction-oriented representations, then amplifies relevant ones and reducing noise from irrelevant ones in a polarity-aware manner to better cater to the enhancement of different spectral information with linear computational complexity. Moreover, considering existing methods' inability to adequately compensate for long distance interactions in reconstruction, we introduce a quant attention paradigm that categorize tokens into semantic-aware groups using an efficient quant operation for attention computation. Experimental results show our method achieves state-of-the-art performance on various simulation datasets and better vision results on real-world datasets.