Empowering Self-Balance of Deep Information Bottleneck for Multimodal Clustering

摘要

Multimodal clustering (MMC) focuses on learning consistent representations through fusing discriminative features from each modality in an unsupervised fashion. Recently, information bottleneck-based MMC methods transform the representation learning into a non-redundant multimodal feature puzzle process. However, they depend on manually setting the trade-off parameter β to compress task-irrelevant features while preserving task-relevant features, which severely impairs final clustering results. In addition, manually setting β overlooks both the information quality balance and the information quantity balance, hindering the learning of compact and meaningful representations for discovering cluster patterns. In this work, we propose a novel βqq-guided information bottleneck (βqq IB) for addressing the aforementioned problems. The core of βqq IB is a self-balance mechanism that consists of a βquality component and a βquantity component. The βquality component aims to achieve an information quality balance by modeling the relation between task characteristics and data sample scale. Meanwhile, the βquantity component intends to realize information quantity balance through modeling the dynamic changes of data scale within the deep variational architect. Experiments on six benchmark datasets demonstrate the superiority of βqq IB method. To the best of our knowledge, this is the first work to investigate self-balance learning in IB-based multimodal clustering.