Temporal-Informative Adapters in VideoMAE V2 and Multi-Scale Feature Fusion for Micro-Expression Spotting-then-Recognize

摘要

Micro-Expression is subtle facial movements that reveal hidden emotions, but their fleeting and involuntary nature poses significant challenges for detection. This paper introduces a novel approach addressing two critical tasks in Micro-Expression analysis: spotting and Recognize. We integrate the VideoMAE V2 framework with a temporal information adapter and multi-scale feature fusion to enhance the performance of Micro-Expression Spotting-then-Recognize. Our method leverages the temporal information adapter to capture local temporal context within video frames, improving feature extraction efficiency. Additionally, we construct a multi-scale image pyramid to capture a range of motion features, from broad movements to subtle details. By combining these multi-scale features, our approach strengthens the model's capabilities in Micro-Expression Spotting-then-Recognize. Our method effectively addresses issues related to environmental variations, involuntary facial movements, and dataset imbalance, leading to improved accuracy in Micro-Expression Spotting-then-Recognize.