With the explosive growth of video-centric social media platforms, understanding and predicting video popularity has become a crucial problem in both academia and industry. This year, the Social Media Prediction (SMP) Challenge expands its scope by introducing a dedicated video track, shifting the focus from static images to dynamic, multimodal video content. We introduce the Social Media Prediction for Videos (SMPV) task and release a large-scale, multimodal benchmark dataset, SMPD-Video, with more than 6K short-form videos, including vision language metadata, user profiles, and popularity labels. This challenge invites global researchers to develop predictive algorithms that integrate spatial-temporal dynamics, multimodal learning, and user-video interactions to forecast video popularity in real-world social temporal streams. With the participation and contribution of top teams around the world, the challenge has seen continuous performance improvements in recent years, driven by technological advancements. SMP Challenge Homepage: www.smp-challenge.com.
论文检索
输入标题、作者或关键词，从 1,620 篇学术成果中精准定位
The proliferation of multimedia content on social media has transformed how information is produced and consumed, enabling real-time coverage of global events but also accelerating the spread of misinformation, particularly during crises such as wars, natural disasters, and elections. The rise of synthetic media and the reuse of authentic content in misleading contexts further underscore the need for robust verification tools. In this paper, we present a comprehensive system developed for the ACM Multimedia 2025 Grand Challenge on Multimedia Verification. Our system evaluates both the authenticity and contextual accuracy of multimedia content in multilingual settings, producing expert-oriented verification reports alongside accessible summaries for the public. We propose a unified verification pipeline that integrates visual forensics, textual analysis, and multimodal reasoning, with a hybrid approach to detecting out-of-context (OOC) media via semantic similarity, temporal alignment, and geolocation cues. Extensive evaluations on the challenge benchmark demonstrate the system's effectiveness across diverse real-world scenarios. Our contributions advance the state of the art in multimedia verification while providing practical tools for journalists, fact-checkers, and researchers addressing information integrity in the digital age.
The pervasive spread of multimedia misinformation and disinformation presents a significant challenge to information integrity, demanding robust and efficient verification methodologies. Accurately assessing the authenticity and context of complex multimedia content across diverse platforms requires advanced analytical capabilities. This work introduces Ægis, a novel AI-enhanced solution developed as the authors' submission to the ACMMM'25 - Grand Challenge on Multimedia Verification, aimed at improving the efficiency of multimedia verification. The proposed solution integrates both state-of-the-art and traditional verification tools to comprehensively address various aspects of verification tasks, including event summarization, forensic analysis, and evidence validation. In addition to selectively applied verification modules, large language models (LLMs) are leveraged extensively to fuse findings, perform reasoning, and generate structured verification reports-significantly streamlining the verification process. Demonstrated on the competition multimedia dataset, Ægis accurately validates content integrity, extracts geospatial and temporal information, and identifies content origin across platforms, offering reliable and ethical support for real-world fact-checking.
This paper presents our submission to the ACMMM25 - Grand Challenge on Multimedia Verification. We developed a multi-agent verification system that combines Multimodal Large Language Models (MLLMs) with specialized verification tools to detect multimedia misinformation. Our system operates through six stages: raw data processing, planning, information extraction, deep research, evidence collection, and report generation. The core Deep Researcher Agent employs four tools: reverse image search, metadata analysis, fact-checking databases, and verified news processing that extracts spatial, temporal, attribution, and motivational context. We demonstrate our approach on a challenge dataset sample involving complex multimedia content. Our system successfully verified content authenticity, extracted precise geolocation and timing information, and traced source attribution across multiple platforms, effectively addressing real-world multimedia verification scenarios.
The 2025 Grand Challenge on Multimedia Verification addresses the challenges of verifying the authenticity and context of online multimedia content. Participants analyzed real-world cases of images and videos, assessed their sources, detected any potential manipulations, and submitted detailed verification reports. The main competition was structured into three stages: Training, Validation, and Real-World Verification, which included live test cases and welcomed solutions ranging from manual methods and OSINT practices to automated tools and novel AI techniques. A total of 32 teams from 11 countries registered. The submitted solutions presented diverse pipelines that integrated forensic analysis, multimodal reasoning, and large language models (LLMs). The results demonstrate significant progress toward semi-automated verification workflows, while also exposing challenges in scalability, consistency, and reliability. In this paper, we present the challenge design, datasets, evaluation criteria, and participant outcomes, providing insights to guide future research and practice in multimedia verification.
A critical challenge in advancing human-like conversational AI systems is enabling models to understand and respond to user emotions contextually, a task known as Multimodal Empathetic Response Generation (MERG). While prevailing multimodal models attempt to resolve cross-modal emotional discrepancies via concatenation or cross-attention, their simplistic fusion mechanisms often fail to account for the nuanced and contradictory nature of human emotions. Consequently, the resulting feature representations suffer from these unresolved internal conflicts, limiting their effectiveness. In this paper, we propose a novel Multimodal Empathetic Reasoning and Inconsistency-Aware (MERIA) framework. MERIA introduces a multimodal disentanglement encoder based on β-VAE and extends the AvaMERG dataset with multimodal chains of empathy (M-CoE). Our framework outperforms existing methods, achieving the best human evaluation scores in the empathetic text response generation task of the MERG 2025 Challenge. Our code is available at https://github.com/DANG-ai/MERIA-MERG.
Emotional Support Chatbots could unlock potential by providing scalable, low-cost, and personal emotional support, overcoming critical accessibility barriers inherent in traditional counseling. However, current text-based Chatbots fall short in conveying the multimodal empathy crucial in counseling. Humans naturally prefer face-to-face communication with peers to share feelings, encompassing spoken tone, micro-expressions, and body language to convey empathy. To bridge this gap, we propose EMO-Avatar, an LLM-agent-orchestrated framework that integrates emotional reasoning capabilities and multimodal expression in counseling. Our approach introduces two innovations: (1) a Multimodal Emotional Support Agent. EMO-Avatar can follow adaptive instruction across TTS, pose, micro-expressions, and body actions, leading to the generation of highly expressive human animations. (2) a Comforting-Exploration-Action support strategy; EMO-Avatar systematically integrates Hill's three-stage counseling theory into its emotional reasoning capability. Guided by the LLM's reasoning, this strategy informs response generation and displays stage-specific preferences for speech, body language, and expressions. EMO-Avatar can provide deeper emotional support and therapeutic human-like interactions. Experimental validation on the AvaMERG Challenge demonstrates EMO-Avatar's superior performance, achieving top-2 ranking among 20 participants across response appropriateness, multimodal consistency, naturalness, and emotional expressiveness metrics. Our demo is available at https://ai4ai.anonymous-demo.fun/.
Multimodal Empathetic Response Generation (MERG) is crucial for building emotionally intelligent human-computer interactions. Although large language models (LLMs) have improved text-based ERG, challenges remain in handling multimodal emotional content and maintaining identity consistency. Thus, we propose E3RG, an Explicit Emotion-driven Empathetic Response Generation System based on multimodal LLMs which decomposes MERG task into three parts: multimodal empathy understanding, empathy memory retrieval, and multimodal response generation. By integrating advanced expressive speech and video generative models, E3RG delivers natural, emotionally rich, and identity-consistent responses without extra training. Experiments validate the superiority of our system on both zero-shot and few-shot settings, securing Top-1 position in the Avatar-based Multimodal Empathy Challenge on ACM MM'25. Our code is available at https://github.com/RH-Lin/E3RG.
The Ava-MERG Challenge at ACM Multimedia 2025 explores avatar-based multimodal empathetic response generation by advancing dialogue systems that respond with emotional awareness across text, speech, and talking-face video. To support this goal, we constructed a high-quality dataset featuring aligned multimodal responses in diverse conversational scenarios. The challenge comprises two progressively complex subtasks centering around multimodal-aware empathetic response generation. We aim to promote research in empathetic AI and affective computing.
Generating diverse and contextually appropriate facial reactions remains a significant challenge due to variability in individual responses, limited explainability, and insufficient modeling of contextual cues. In this study, we propose a multimodal framework that integrates behavioral memory, dynamic attention control, and cognitive style modeling to generate personalized and psychologically grounded facial reactions in dyadic interactions. Our method models the causal link between speaker behavior and listener response by incorporating frame-level behavioral cues, personality traits, and cognitive processing styles. The proposed system consists of three core components: a behavioral memory module that captures temporal context across conversation turns; a Personalized Personality Recognition Style (PPRS) module that infers cognitive tendencies via dual-path learning based on the Big Five personality traits; and a transformer-based generative module equipped with diffusion modeling and context-aware attention gating. This design enables the generation of expressive, individualized responses even during silence or scene transitions. We conduct extensive evaluations on the REACT2025 benchmark using the MARS dataset. Results show that our method outperforms state-of-the-art models in appropriateness (FRCorr ↑0.71), diversity (FRDiv ↑0.1405), and synchrony (FRSyn ↑47.77), ranking 1st in the offline track and 3rd in the online setting. These findings highlight the framework's effectiveness in simulating human-like, emotionally congruent reactions while offering interpretability grounded in personality psychology.
Generating multiple appropriate facial reactions (MAFR) is essential for effective human-agent interaction. However, existing methods typically do not jointly model both local and global emotional cues, and neglect the temporal dynamics of facial expressions, leading to emotionally inconsistent and less natural reactions. In this work, we combine local and global emotional features to form a more comprehensive emotional representation. Our method further introduces motion-aware visual features that capture the dynamic evolution of facial expressions beyond static frames. By integrating both appearance and motion information within a structured generative framework, our approach enables more context-aware and temporally natural listener reactions. Experimental results demonstrate that our method outperforms existing approaches in both reaction diversity and appropriateness, which ranked first in the React 2025 challenge offline track.The implementation code can be accessed at:https://github.com/mtv-2025-react/mtv-2025-2025.git.
As embodied intelligence has become a new hot topic in current artificial intelligence research, facial reaction generation has increasingly become a key technology for achieving natural human-computer interaction. Existing methods typically rely on bimodal inputs of audio and visual signals, but they still suffer from poor cross-modal consistency and insufficient feature fusion, making it difficult to effectively capture complex facial features. To enhance the representational capacity of fused features, this paper proposes a Scattering-Conditioned Diffusion Model (SC-Diff), which extracts stable multi-scale structural features in the frequency domain via wavelet scattering transform and injects them into the diffusion generation process as conditional information, thereby enhancing the modeling ability of representing local facial variations. Furthermore, considering that different prediction tasks exhibit varying sensitivity to target changes during training, we introduce an uncertainty-based adaptive loss weighting strategy to dynamically balance three types of supervision targets: facial action units, facial affect, and facial expressions. Experimental results on the REACT 2025 dataset demonstrate that the proposed method outperforms existing state-of-the-art approaches across multiple evaluation metrics.
In dyadic interactions, a broad spectrum of human facial reactions might be appropriate for responding to each human speaker behaviour. Following the successful organisation of the REACT 2023 and REACT 2024 challenges, we are proposing the REACT 2025 challenge encouraging the development and benchmarking of Machine Learning (ML) models that can be used to generate multiple appropriate, diverse, realistic and synchronised human-style facial reactions expressed by human listeners in response to an input stimulus (i.e., audio-visual behaviours expressed by their corresponding speakers). As a key of the challenge, we provide challenge participants with the first natural and large-scale multi-modal Multiple Appropriate Facial Reaction Generation (MAFRG) dataset (called MARS) recording 136 human-human dyadic interactions containing a total of 2856 interaction sessions covering five different topics. In addition, this paper also presents the challenge guidelines and the performance of our baselines on the two proposed sub-challenges: Offline MAFRG and Online MAFRG, respectively. The challenge baseline code is publicly available at https://github.com/reactmultimodalchallenge/baseline_react2025
This paper presents our solution for the Micro-Expression Visual Question Answering (ME-VQA) task in the 2025 Facial Micro-Expression Grand Challenge (MEGC). To address the limitations of traditional micro-expression recognition (MER) methods in dynamic modeling, semantic interpretation, and natural language interaction, we propose Emotion-Qwen-VL, a fully fine-tuned multimodal large language model tailored for micro-expression understanding. Specifically, we construct a structured, instruction-based QA dataset that reformulates emotion categories and action unit (AU) annotations into natural language QA pairs, covering classification, AU detection, and causal reasoning. We then adopt a full-parameter fine-tuning strategy to guide Qwen2.5-VL in learning fine-grained temporal facial dynamics and their emotional semantics. Experimental results on the MEGC 2025 test set demonstrate that Emotion-Qwen-VL outperforms strong baselines such as Qwen2.5-VL and QVQ across multiple dimensions, including coarse-grained macro-expression classification, fine-grained micro-expression classification, and language generation. Our results highlight the effectiveness, interpretability, and adaptation potential of large models in micro-expression understanding. The code is available at: https://github.com/2308623956/MEGC2025.
Micro-expressions (MEs) are involuntary, low-intensity, and short-duration facial expressions that often reveal an individual's genuine thoughts and emotions. Most existing ME analysis methods rely on window-level classification with fixed window sizes and hard decisions, which limits their ability to capture the complex temporal dynamics of MEs. Although recent approaches have adopted video-level regression frameworks to address some of these challenges, interval decoding still depends on manually predefined, window-based methods, leaving the issue only partially mitigated. In this paper, we propose a prior-guided video-level regression method for ME analysis. We introduce a scalable interval selection strategy that comprehensively considers the temporal evolution, duration, and class distribution characteristics of MEs, enabling precise spotting of the onset, apex, and offset phases. In addition, we introduce a synergistic optimization framework, in which the spotting and recognition tasks share parameters except for the classification heads. This fully exploits complementary information, makes more efficient use of limited data, and enhances the model's capability. Extensive experiments on multiple benchmark datasets demonstrate the state-of-the-art performance of our method, with an STRS of 0.0562 on CAS(ME)3 and 0.2000 on SAMMLV. The code is available at https://github.com/zizheng-guo/BoostingVRME.
The rise of Multimodal Large Language Models (MLLMs) offers new opportunities for Micro-Expression (ME) analysis. This paper introduces Micro-Expression Visual Question Answering (ME-VQA), a novel task reformulating ME annotations (e.g., emotion categories, action units) into QA pairs. To address key challenges-hardware limitations, context inconsistency, and compositional reasoning gaps-we propose a Relationship-Aware Hierarchical VQA Framework. Our approach leverages mined emotion correlations (e.g., coarse-to-fine label dependencies) and employs a two-stage process: 1) Coarse-grained anchoring for broad emotion categories, and 2) Fine-grained reasoning constrained by coarse outputs and statistical rules. We further optimize efficiency via a dual-phase video sampling strategy: during training, keyframes (onset/apex/offset) and random non-expression frames are used; uniform sampling is applied at inference. Experiments demonstrate significant improvements in answer consistency and accuracy.
Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. In recent years, substantial advancements have been made in the areas of ME recognition, spotting, and generation. However, conventional approaches that treat spotting and recognition as separate tasks are suboptimal, particularly for analyzing long-duration videos in realistic settings. Concurrently, the emergence of multimodal large language models (MLLMs) and large vision-language models (LVLMs) offers promising new avenues for enhancing ME analysis through their powerful multimodal reasoning capabilities. The ME grand challenge (MEGC) 2025 introduces two tasks that reflect these evolving research directions: (1) ME spot-then-recognize (ME-STR), which integrates ME spotting and subsequent recognition in a unified sequential pipeline; and (2) ME visual question answering (ME-VQA), which explores ME understanding through visual question answering, leveraging MLLMs or LVLMs to address diverse question types related to MEs. All participating algorithms are required to run on this test set and submit their results on a leaderboard. More details are available at https://megc2025.github.io.
Major Depressive Disorder (MDD) is a prevalent and severe psychiatric disorder, and its detection remains challenging due to the complexity and variability of its symptoms. Traditional single-modality methods often fail to capture the full spectrum of depressive cues, which has led to the rise of multimodal methods. The ACM Multimedia 2025 ''Multimodal Personality-Aware Depression Detection Challenge'' (MPDD 2025) aims to advance the development of more accurate depression detection models by incorporating multimodal data. In this paper, we proposed a Multi-Level Segment Fusion Based on Adaptive Time-Window Selection (MSF-ATS) method for the MPDD-Elderly Track. To address the challenge of sparse and transient depressive symptoms, we fuse segment-level classifications to obtain subject-level classifications. An adaptive time-window selection based on mean class variance is employed to choose the window with the smallest variance for more stable detection results. Our method achieved an average score of 0.8576 on the MPDD 2025 official test set, significantly outperforming the baseline score of 0.6675.
Depression detection remains challenged by generalized modeling approaches that fail to account for individual heterogeneity. To address this, the Multimodal Personality-aware Depression Detection (MPDD) Challenge introduced personalized features into the modeling process, aiming to better capture individual variability. However, the baseline models still exhibit two critical limitations: the neglect of textual semantics embedded in audio, and inconsistent predictions for the same subject across tasks and samples. Motivated by these limitations, we introduce HOPE (Hierarchical fusion for Optimized and Personality-aware Estimation of Depression), a unified framework for consistent, subject-level depression estimation. HOPE first employs a Latent Semantic Projection (LSP) module to reconstruct textual semantics from audio features when transcripts are unavailable. It then introduces a consistency-aware integration mechanism that hierarchically fuses multi-branch predictions to resolve inter-task and inter-sample contradictions. HOPE achieved first place in the MPDD Challenge Young Track, demonstrating strong cross-modal learning capabilities and consistent, subject-level depression prediction.
Depression is an increasingly prevalent mental health issue worldwide, especially among the elderly, where effective early detection is crucial for timely intervention. While recent multimodal approaches demonstrate promise in leveraging visual and auditory cues for automatic depression recognition, existing methods often fail to extract fine-grained, depression-specific patterns from pre-extracted features and underutilize cross-modal interactions. To address these challenges, we propose DepFormer, a unified framework that incorporates a Bimodal Collaborative Transformer(BCT) for cross-modal representation learning and a personalized fusion module to enhance individual-specific modeling. The architecture comprises: (1) unimodal feature extraction, (2) bimodal collaborative representation learning via the BCT, (3) personalized feature fusion, and (4) final depression classification. Critically, the BCT employs symmetric bidirectional branches comprising an Audio-to-Video Transformer and a Video-to-Audio Transformer to enable mutual enhancement and complementary learning across modalities. Extensive experiments validate DepFormer's effectiveness, securing first place in the MPDD Challenge 2025 (Elderly Track), underscoring its strong practical potential.