TAR: Token-Aware Refinement for Fine-grained Generalized Category Discovery

摘要

For an unlabeled dataset containing both known and unknown categories, Generalized Category Discovery (GCD) aims to classify known categories while discovering unknown ones. Although existing methods achieve promising results on coarse-grained datasets, they struggle in fine-grained scenarios. We observe that attention artifacts where attention maps exhibit abnormally high responses on a few tokens--significantly interfere with fine-grained GCD by causing the model to overemphasize global semantics and overlook discriminative local cues. To address this issue, we propose the Token-Aware Refinement (TAR) framework, a plug-and-play module that mitigates attention artifacts and enhances local information. Unlike conventional approaches that rely solely on the first token for classification, TAR leverages the entire token sequence to better capture fine-grained details. Extensive experiments demonstrate that TAR consistently improves performance across multiple benchmarks. Our code is available at https://github.com/VectorYangYiStar/TAR.