Dual Uncertainty-Guided Feature Alignment Learning for Text-Based Person Retrieval

摘要

Text-based person retrieval (TBPR) aims to retrieve pedestrian images based on textual descriptions, facing challenges due to the inherent heterogeneity and uncertainty between visual and textual modalities. Most existing methods focus on addressing heterogeneity while neglecting the issue of uncertainty. To tackle the uncertainty arising from the diverse textual expressions, including both structural and semantic content variations, we propose a novel Dual Uncertainty-Guided Feature Alignment Learning (DUAL) approach, utilizing instance-level and identity-level uncertainty estimations to mitigate these impacts. Specifically, for the uncertainty caused by textual structure variations, DUAL first introduces an uncertainty Gaussian modeling module that represents image and text features as Gaussian distributions in a learnable manner, and estimates instance-level uncertainty coefficients to quantify structural differences within the text. Subsequently, DUAL leverages ShareGPT4V to standardize the text structure, dynamically aligning the original text features with structure-invariant generated text features through adaptive knowledge distillation guided by the instance-level uncertainty coefficients, effectively reducing structural diversity's impact while minimizing noise. Moreover, for the uncertainty caused by the diversity of textual semantic content, DUAL designs an alignment loss that utilizes identity-level uncertainty coefficients, estimated via a Gaussian Mixture Model based on the distances between image and text features of the same identity, effectively mitigating the impact of semantic content diversity. Experimental results demonstrate that DUAL outperforms existing methods on TBPR benchmarks, highlighting its superiority in multimodal person retrieval.