CDIB: Consistency Discovery-guided Information Bottleneck for Multi-modal Knowledge Graph Reasoning

摘要

Multi-modal knowledge graph reasoning (MKGR) seeks to conjecture plausible facts in MKGs by learning effective representations from various modalities (e.g., structure, text, and image). However, due to holistic redundancy (i.e., each modality carries task-irrelevant redundancy) and modality conflict (i.e., different modalities contain contradictory information), the reasoning performance of current methods is substantially impaired. In this paper, we propose a novel Consistency Discovery-guided Information Bottleneck (CDIB) framework to address the aforementioned challenges. Specifically, a modality compression module is first designed to learn modality-private entity representations of alleviating redundant information. Then, a consistency discovery module is developed to discover cross-modal consistency during multi-modal fusion to learn the comprehensive entity representations. To retain task-relevant information, an information preservation module is devised to further enrich the comprehensive entity representations to be predictive for MKGR. Extensive experiments indicate that CDIB achieves state-of-the-art reasoning ability on two benchmark datasets over current MKGR baselines, and also exhibits promising robustness against noise.