Multimodal Decomposed Distillation with Instance Alignment and Uncertainty Compensation for Thermal Object Detection

摘要

RGB-Thermal images leverage complementary optical and thermal modalities to identify objects. While achieving superior performance, the reliance on multimodal fusion inherently limits inference efficiency and adaptability to harsh RGB-failure environments. In this work, we propose a multimodal decomposed distillation framework to develop robust thermal-only detectors by transferring knowledge from multimodal teachers. Unlike conventional one-to-one distillation, we decouple the tasks of simultaneously mimicking RGB-T teacher representations and preserving thermal-specific student feature integrity into dual branches to avoid intrinsic semantic conflicts. Specifically, we present channel-adaptive prompt learning for cross-modal decomposition and a frequency-guided dynamic module for decomposed knowledge integration. The dual-branch architecture employs asymmetric training objectives to ensure effective cross-modal knowledge transfer while preserving the integrity of thermal information. Furthermore, to exploit finer-grained instance knowledge across both feature and prediction levels, we introduce a customized instance alignment distillation to enhance the local discriminability in feature pyramids, and propose an uncertainty-aware logit distillation to compensate for ambiguous predictions in detection heads. Experiments on three datasets validate the effectiveness of our framework in boosting thermal-based detectors. Code is released at https://github.com/lyf0801/DecomKD.