DDFD: Diffusion-Based Denoising Fusion for Object Detection in Infrared-Visible Images

摘要

Infrared-visible image fusion for object detection (IVIF-OD) aims to utilize complementary information in the two modalities to synthesize new images with richer information to serve object detection. Most existing works focus on how to better fuse pixel-level details while ignoring object-related information required for detection and introducing redundant and object-irrelevant information in the fused images. To address the limitations of previous studies, this paper proposes a diffusion-based denoising fusion for object detection in infrared-visible images, termed DDFD. Specifically, DDFD treats image fusion as a diffusion-based denoising process to generate fused images that are informative yet non-redundant. Since visible imaging is easily affected by adverse conditions, DDFD exploits an image-adaptive enhancement (IAE) module that adaptively improves visible images to achieve better fusion. To extract key fusion features and remove redundancy, DDFD uses an image-aware noise estimator (INE) to determine the noise in the input infrared-visible images for promoting the diffusion denoising network. To take advantage of both the fusion network and object detection network, DDFD jointly optimizes them such that the fusion network can receive object information to improve the fused images, and the improved images can provide high-quality features to enhance object detection performance. Extensive experiments on the M3FD, DroneVehicle, and VEDAI public datasets reveal the superior object detection performance of DDFD and confirm the effectiveness of IVIF-based object detection under challenging weather conditions.