Progressive Large-Scale Modeling via Temporal-Spatial Focus Connector for Micro-Action Recognition

摘要

Recent advances in video action recognition have achieved remarkable performance in coarse-grained macro-action classification by leveraging large-scale visual backbones and transformer architectures. However, extending these successes to fine-grained micro-action recognition remains a fundamental challenge due to the subtlety, brevity, and low motion intensity of micro-actions. In this paper, we propose a high-capacity framework for micro-action recognition, enhancing both representation learning and decision robustness. We scale to large-scale backbones using the VideoMAEv2 Giant model, enabling the extraction of finer spatial-temporal features. A Temporal-Spatial Connector (TSC) is introduced to dynamically highlight discriminative temporal frames and spatial regions, strengthening the model's focus on subtle motion cues critical for micro-action identification. To stabilize optimization and fully exploit the capacity of large models, we design a four-phase progressive training strategy, encompassing linear probing, full fine-tuning, connector-specific optimization, and classifier head refinement. Furthermore, we propose a novel ensemble decision mechanism that integrates Top-K predictions from diverse models via a Large Language Model (LLM), enhancing prediction consistency and robustness through multimodel consensus. Our method achieves an F1mean of 76.54% on the MA-52 dataset, ranking 3rd in the 2025 Micro-Action Analysis Grand Challenge and advancing the state of the art in fine-grained video understanding.