GPT-ReID: Learning Fine-grained Representation with GPT for Text-based Person Retrieval

摘要

Text-based Person Retrieval (TBPR) is a challenging task that aims to retrieve pedestrian images according to natural language descriptions. Existing works mainly focus on discriminative feature learning via exploring cross-modal matching methods, while the overfitting issues caused by insufficient labeled data and the absence of well-designed auxiliary tasks are often overlooked. Motivated by the recent progress of large language models (LLMs), we propose a novel method named GPT-ReID for TBPR, which aims to leverage the strong comprehension of LLMs to alleviate the overfitting risk. Specifically, based on the great power of GPT, GPT-ReID first introduces an adversarial text generation scheme called GPTGAN, which aims to generate comprehensive strong positive captions and deceptive hard negative captions through the original captions for a single image. Furthermore, a joint auxiliary learning strategy is also proposed which contains Multi-Relation Aware (MRA), Keywords Masked Language Model (KMLM), and Keywords Replacement Detection (KRD), to facilitate global- and token-level optimization, enhancing cross-modal granular representation alignment. Extensive experiments on a set of highly competitive benchmark datasets validate the merits of the proposed GPT-ReID against a flurry of state-of-the-art methods, with Rank-1 accuracy reaching 78.42%, 69.43%, and 70.06% on CUHK-PEDS, ICFG-PEDS, and RSTPReid, respectively.