TFPA: Text Features Guided Dynamic Parameter Adjustment for Few Shot Action Recognition

摘要

Most few-shot learning methods aim to train models to learn parameters that can generalize to new categories using training sets, after which the model parameters are typically fixed. However, due to limited data, models often fail to learn generalizable parameters, as they tend to overfit source domain-specific inductive biases. This can lead to catastrophic forgetting or poor adaptation to new domains. Unlike previous methods, we propose a Text Feature guided dynamic Parameter Adjustment (TFPA) method for few-shot action recognition. Inspired by basis decomposition in vector spaces, TFPA reformulates the traditional linear layer into a set of basis mapping matrices in the parameter space. Each matrix functions analogously to a basis vector in linear algebra, and their linear combinations collectively span the parameter space. To construct a domain-adaptive parameter matrix from these combinations, we propose a Coordinate Vector Computation (CVC) module, which leverages text features as semantic guidance to adaptively estimate optimal linear combination coefficients for the basis mapping matrices. Furthermore, we propose a Centroid Exclusion Loss (CEL) and a Contrastive Clustering Loss (CCL) to enhance the distinctiveness among the basis mapping matrices. These regularization terms promote functional specialization and reduce redundancy across the basis mapping matrices, thereby enhancing performance. Experimental results on five benchmark datasets demonstrate the effectiveness and strong generalization ability of our method in few-shot action recognition. The code will be released soon at https://github.com/ReverseSuzhou/TFPA.