Imagining Vision From Language for Few-Shot Class-Incremental Learning

摘要

Few-Shot Class-Incremental Learning (FSCIL) aims to enable the model to continuously learn new categories with only limited samples and maintain the recognition ability of old categories. However, this task faces two core challenges: catastrophic forgetting and overfitting due to data scarcity. Existing methods mostly rely on fine-tuning strategies or the transfer ability of visual language models, which is difficult to fundamentally solve the problem of insufficient image samples. To this end, we propose an incremental learning method called Imagining Vision From Language (IVFL), which consists of a base session and multiple incremental sessions. In the base session, the model generates imagined visual features through language descriptions, learns the mapping relationship from language to vision, and estimates the feature distribution of each category; in the incremental session, the model jointly uses old-class pseudo-image features, new-class images, and language descriptions to guide the model to perform effective category expansion while maintaining old-class knowledge. Our IVFL achieves an effective balance between new and old knowledge and achieves significant performance improvements on miniImageNet, CIFAR100, and CUB200 datasets, verifying its superiority in scenarios with data scarcity and knowledge retention.