Agent-MER: A Cognitive Agent with Hierarchical Deliberation for Open-Vocabulary Multimodal Emotion Recognition

摘要

This paper focuses on Open-Vocabulary Multimodal Emotion Recognition (OV-MER) and is dedicated to solving the two challenges it faces: concept semantic misalignment and incomplete coverage of fine-grained emotion categories. To address this, we propose a novel cognitive agent framework (Agent-MER), which reframes the OV-MER task as a problem to be solved by an agent that mimics the human cognitive process through knowledge-guided deliberation. We first construct a hierarchical Emotion Tree to serve as the agent's knowledge base. Building on this, we design a Knowledge-Guided Hierarchical Deliberation reasoning process. This process systematically explores the entire emotional landscape through a three-level, coarse-to-fine iterative reasoning process, enabling the identification of a richer and deeper range of emotions. Finally, a Self-Consistent Voting mechanism is employed to aggregate the results from multiple reasoning runs, ensuring the robustness of the final output. Experiments conducted in the MER2025 Challenge demonstrate that our proposed method achieved a top-ranking score of 61.04%, securing first place and significantly outperforming existing baselines. This work not only provides an effective solution for OV-MER but also opens up new avenues for developing more human-like affective intelligence systems.