We introduce the MIRAGE Challenge, a comprehensive benchmark for multimodal interleaved reasoning and generation, to ACM MM 2025. The challenge aims to evaluate models' abilities to both understand and generate content from complex, multimodal contexts consisting of interlinked images and text. The challenge is accompanied by the MIRAGE Dataset, comprising 263.7K high-quality instruction-response pairs across 35 tasks in two tracks: reasoning and generation. These pairs span 20 diverse scenarios, from surveillance to artistic creation, ensuring broad coverage. The challenge includes seven major categories: Multi-Image Reasoning, Document and Knowledge-Based Understanding, Interactive Multi-Modal Communication, Multi-Image Discrimination, Sequential Visual Generation, Material-based Image Coloring, and Visual Reference Customization. Hosting the MIRAGE Challenge at MM 2025 will drive significant progress in unified multimodal learning and inspire broad involvement in developing more versatile AI systems capable of both understanding and generating multimodal content. Challenge details and participation information are available at https://mm25mirage.github.io/mirage/.
论文检索
输入标题、作者或关键词，从 1,620 篇学术成果中精准定位
LAVA Challenge 2025 aims to improve the ability of large visual language models to accurately understand complex visual information such as data flow diagrams and Gantt charts contained in Japanese government and business documents. For this challenge, we adopted a two-stage approach consisting of retrieval and reading comprehension. Specifically, in the retrieval step, we select pages relevant to the question from multi-page PDF documents, and in the reading comprehension step, we perform question answering by referring to the top k images selected in the retrieval step. For the retrieval step, we employ ColQwen2, which performs visual information retrieval using the multilingual Qwen2-VL. For the reading comprehension step, we propose a method that performs question answering, including voting, using the multilingual visual language model Qwen2.5VL under different prompts, model sizes, and image qualities. In LAVA Challenge 2025, we clarify the importance of the two-step process of retrieval search and reading comprehension in visual question answering, and verify the effectiveness of a method for determining the multiple results of reading comprehension steps through voting.
Multimodal Large Language Models (MLLMs) have demonstrated remarkable multimodal understanding capabilities in Visual Question Answering (VQA) tasks by integrating visual and textual features. However, under the challenging ten-choice question evaluation paradigm, existing methods still exhibit significant limitations when processing PDF documents with complex layouts and lengthy content. Notably, current mainstream models suffer from a strong bias toward English training data, resulting in suboptimal performance for Japanese and other language scenarios. To address these challenges, this paper proposes a novel Japanese PDF document understanding framework that combines multimodal hierarchical reasoning mechanisms with Colqwen-optimized retrieval methods, while innovatively introducing a semantic verification strategy through sub-question decomposition. Experimental results demonstrate that our framework not only significantly enhances the model's deep semantic parsing capability for complex documents, but also exhibits superior robustness in practical application scenarios.
Document Visual Question Answering (Document VQA) faces significant challenges when processing long documents in low-resource environments due to context limitations and insufficient training data. This paper presents AdaDocVQA, a unified adaptive framework addressing these challenges through three core innovations: a hybrid text retrieval architecture for effective document segmentation, an intelligent data augmentation pipeline that automatically generates high-quality reasoning question-answer pairs with multi-level verification, and adaptive ensemble inference with dynamic configuration generation and early stopping mechanisms. Experiments on Japanese document VQA benchmarks demonstrate substantial improvements with 83.04% accuracy on Yes/No questions, 52.66% on factual questions, and 44.12% on numerical questions in JDocQA, and 59% accuracy on LAVA dataset. Ablation studies confirm meaningful contributions from each component, and our framework establishes new state-of-the-art results for Japanese document VQA while providing a scalable foundation for other low-resource languages and specialized domains. Our code available at: https://github.com/Haoxuanli-Thu/AdaDocVQA.
The advent of Large Vision-Language Models (LVLMs) has demonstrated significant capabilities in multimodal understanding. However, their application to complex, multi-page documents, particularly in non-English languages like Japanese, remains a significant challenge due to the scarcity of suitable benchmarks. To address this gap, we organized the ''Large Vision---Language Model Learning and Applications (LAVA) Grand Challenge'' at ACM MultiMedia 2025. We present an overview of the competition. We designed a novel, challenging task: a 10-way multiple-choice Visual Question Answering (VQA) task on multi-page Japanese PDF documents. The task demands that models integrate information across multiple pages, text, and figures. We detail the dataset construction, including an annotation and filtering process designed to ensure questions are visually grounded and non-trivial. We also present the competition results, including an analysis of the leaderboard, and discuss the baseline performance of representative models. The LAVA Grand Challenge highlighted both the current capabilities and limitations of LVLMs in practical document understanding scenarios, thereby stimulating future research and providing a robust benchmark in this important domain.
Recently, MGTV organized the Image-to-Video Model Acceleration Challenge, calling for participants to propose optimization solutions for the Wan 2.1-14B model. The challenge emphasizes techniques such as quantization and GPU acceleration to improve the model's inference efficiency. As AIGC technology advances rapidly, video generation large models exhibit great potential in content creation, yet they face critical challenges of high computing power consumption, long inference time, and excessive VRAM usage during inference, which severely hinder content production efficiency. This challenge aims to explore approaches for efficient video generation under limited computing resources, requiring participants to reduce the model's computing power and VRAM demands while improving inference speed, all without compromising generation quality. To support participants' development and evaluation, the challenge provides a baseline framework and test dataset. For further details, please refer to the official challenge website (https://challenge.ai.mgtv.com/#/track/53).
Identity preservation is a critical capability in video generation and one of the core requirements for high-quality video synthesis. Existing approaches typically extract facial features from reference images as conditional inputs and inject them into the generation pipeline to maintain subject identity. However, in the IPVG Challenge 2025, state-of-the-art models such as ConcatID still fall short of delivering satisfactory identity preservation. To address this limitation, we propose a simple yet highly effective multi-branch video generation framework based on entity routing. Concretely, we integrate several fine-tuned dedicated models to compensate for the base model's weaknesses in identity preservation, dynamically selecting the appropriate branch according to each prompt. In addition, we employ enhanced prompts to further steer the generation process. Remarkably, using just a single NVIDIA RTX 3090 GPU for 120 hours of training, we boost the baseline's cur_score from 0.242 to 0.313.
Identity-preserving text-to-video (IPT2V) generation creates videos faithful to both a reference subject image and a text prompt. While fine-tuning large pretrained video diffusion models on ID-matched data achieves state-of-the-art result on IPT2V, data scarcity and high tuning costs hinder broader improvement. We thus introduce a Training-Free Prompt, Image, and Guidance Enhancement (TPIGE) framework that bridges the semantic gap between the video description and the reference image and design sampling guidance that enhance identity preservation and video quality, achieving performance gains at minimal cost. Specifically, we first propose 1 Face Aware Prompt Enhancement, using GPT-4o to enhance the text prompt with facial details derived from the reference image. We then propose 2 Prompt Aware Reference Image Enhancement, leveraging an identity-preserving image generator to refine the reference image, rectifying conflicts with the text prompt. The above mutual refinement significantly improves input quality before video generation. Finally, we propose 3 ID-Aware Spatiotemporal Guidance Enhancement, utilizing an unified gradients to optimize identity preservation and video quality jointly during generation. Our method outperforms prior work and is validated by automatic and human evaluations on a 1000-video test set-winning first place in the ACM Multimedia 2025 Identity-Preserving Video Generation Challenge, demonstrating state-of-the-art performance and strong generality. The code is available at https://github.com/Andyplus1/IPT2V.git
Identity-preserving text-to-video (IPT2V) generation, which aims to create high-fidelity videos with consistent human identity, has become crucial for downstream applications. However, current end-to-end frameworks suffer a critical spatial-temporal trade-off: optimizing for spatially coherent layouts of key elements ( e.g., character identity preservation) often compromises instruction-compliant temporal smoothness, while prioritizing dynamic realism risks disrupting the spatial coherence of visual structures. To tackle this issue, we propose a simple yet effective spatial-temporal decoupled framework that decomposes representations into spatial features for layouts and temporal features for motion dynamics. Specifically, our paper proposes a semantic prompt optimization mechanism and stage-wise decoupled generation paradigm. The former module decouples the prompt into spatial and temporal components. Aligned with the subsequent stage-wise decoupled approach, the spatial prompts guide the text-to-image (T2I) stage to generate coherent spatial features, while the temporal prompts direct the sequential image-to-video (I2V) stage to ensure motion consistency. Experimental results validate that our approach achieves excellent spatiotemporal consistency, demonstrating outstanding performance in identity preservation, text relevance, and video quality. By leveraging this simple yet robust mechanism, our algorithm secures the runner-up position in 2025 ACM Multimedia Challenge. Our code is available at https://github.com/rain152/IPVG.
Recent advancements in multimodal AIGC have enabled impressive text-to-video synthesis, but a critical challenge remains: maintaining consistent identity of key subjects across generated frames. To address this limitation, we introduce the Identity-Preserving Video Generation (IPVG) grand challenge. This challenge aims to propel the field toward more controllable generative models by focusing community efforts on preserving identity during the video generation process. To support these efforts, we publicly release the Identity-Preserving Video Benchmark (VIP-200K), a novel dataset comprising approximately 500,000 video-prompt pairs with 200,000 unique identities, each coupled with a reference identity image. Through this grand challenge and dataset, we provide a fertile ground for developing solutions that lead to more user-steerable video synthesis systems. The challenge homepage is https://hidream-ai.github.io/ipvg-challenge.github.io/.
Despite recent advances, deepfake detectors remain vulnerable to adversarial examples, particularly in diverse, real-world settings. We propose MIG-COW, a novel adversarial attack framework that generates highly generalizable and visually imperceptible adversarial examples. By combining momentum-integrated gradients with a consensus-orthogonal decomposition, MIG-COW captures both shared and model-specific vulnerabilities across heterogeneous CNN and ViT detectors. On the AADD-2025 Challenge benchmarks, MIG-COW achieves a 99.96% white-box attack success rate (ASR) with high perceptual similarity (SSIM), significantly outperforming existing baselines. However, its limited 7.16% ASR against official black-box targets-despite achieving the best overall score-highlights the ongoing challenge of transferability. We also demonstrate that incorporating low-performing but diverse models in the ensemble can degrade attack effectiveness, underscoring the need for careful surrogate model selection in real-world adversarial settings.
Due to their powerful image generation capabilities, diffusion-based adversarial example generation methods through image editing are rapidly gaining popularity. However, due to reliance on the discriminative capability of the diffusion model, these diffusion-based methods often struggle to generalize beyond conventional image classification tasks, such as in Deepfake detection. Moreover, traditional strategies for enhancing adversarial example transferability are challenging to adapt to these methods. To address these challenges, we propose a unified framework that seamlessly incorporates traditional transferability enhancement strategies into diffusion model-based adversarial example generation via image editing, enabling their application across a wider range of downstream tasks. Our method won first place in the ''1st Adversarial Attacks on Deepfake Detectors: A Challenge in the Era of AI-Generated Media'' competition at ACM MM25, which validates the effectiveness of our approach.
The rapid development of generative AI and in particular deepfake technology enables the seamless creation and manipulation of visual content. As the resulting syntheses are often indistinguishable from authentic images, they threaten the integrity of visual evidence. While forensic detectors can be used to detect syntheses, they can become targets of adversarial attacks. In the ''Adversarial Attacks on Deepfake Detectors'' challenge, competitors were tasked with perturbing a dataset of AI-synthesized images so that four classifiers would mistakenly accept them as authentic. In this paper, we introduce our solution, a white-box adversarial framework that injects globally distributed, data-driven noise perturbations optimized via additional surrogate Vision Transformer and EfficientNet classifiers. Empirical comparisons to both conventional post-processing transforms and localized adversarial patches demonstrate that our approach based on globally distributed noise achieves the highest attack success rates across all public detectors while preserving superior SSIM, confirming its efficacy and visual imperceptibility. In the final evaluation of the challenge, our proposed approach placed third with a final score of 2679.
The rapid proliferation of AI-generated media, particularly hyper-realistic deepfakes, has underscored the critical need for robust detection systems to mitigate risks such as misinformation and identity theft. However, state-of-the-art deepfake detectors remain vulnerable to adversarial attacks-subtle perturbations designed to evade classification. To address this gap, we organized the Adversarial Attacks on Deepfake Detectors (AADD-2025) challenge, a competitive evaluation aimed at advancing methodologies to expose and strengthen weaknesses in deepfake detection models. The challenge tasked participants with generating adversarial examples capable of evading four diverse classifiers (including ResNet, DenseNet, and two blind models) while preserving structural similarity to original deepfakes. A dataset comprising 16 subsets of high- and low-quality deepfake images generated by GAN-based and diffusion models (e.g., StableDiffusion, StyleGAN3) was provided. Participants were evaluated using a weighted combination of Structural Similarity Index (SSIM) and attack success rates across all classifiers. Thirteen teams proposed innovative solutions leveraging techniques such as latent-space manipulation, ensemble gradient optimization, surrogate modeling, and frequency-domain perturbation. Top-performing approaches, including MR-CAS (1st place), Safe AI (2nd place), and RoMa (3rd place), achieved high SSIM scores (0.74-0.93) while successfully misleading classifiers. Notably, MR-CAS's latent diffusion model inversion strategy and Safe AI's consensus-orthogonal gradient weighting framework demonstrated superior transferability across architectures, including Vision Transformers. The challenge revealed critical insights: latent-space attacks outperformed pixel-level methods, ensemble-based strategies enhanced cross-model robustness, and adversarial perturbations optimized for both CNNs and transformers proved most effective. However, gaps persist in generalizing attacks across heterogeneous models and maintaining perceptual fidelity, highlighting the urgency of developing adaptive defenses and hybrid detection mechanisms. By fostering collaboration and innovation, AADD-2025 provides a benchmark for evaluating adversarial robustness in deepfake detection and underscores the need for resilient systems in the era of AI-generated media.
The rapid development of audio-driven talking head generators and advanced Text-To-Speech (TTS) models has led to more sophisticated temporal deepfakes. These advances highlight the need for robust methods capable of detecting and localizing deepfakes, even under novel, unseen attack scenarios. Current state-of-the-art deepfake detectors, while accurate, are often computationally expensive and struggle to generalize to novel manipulation techniques. To address these challenges, we propose multimodal approaches for the AV-Deepfake1M 2025 challenge. For the visual modality, we leverage handcrafted features to improve interpretability and adaptability. For the audio modality, we adapt a self-supervised learning (SSL) backbone coupled with graph attention networks to capture rich audio representations, improving detection robustness. Our approach strikes a balance between performance and real-world deployment, focusing on resilience and potential interpretability. On the AV-Deepfake1M++ dataset, our multimodal system achieves AUC of 92.78% for deepfake classification task and IoU of 0.3536 for temporal localization using only the audio modality.
The field of visual and audio generation is burgeoning with new state-of-the-art methods. This rapid proliferation of new techniques underscores the need for robust solutions for detecting synthetic content in videos. In particular, when fine-grained alterations via localized manipulations are performed in visual, audio, or both domains, these subtle modifications add challenges to the detection algorithms. This paper presents solutions for the problems of deepfake video classification and localization. The methods were submitted to the ACM 1M Deepfakes Detection Challenge, achieving the best performance in the temporal localization task and a top four ranking in the classification task for the TestA split of the evaluation dataset.
Advances in Generative AI have made video-level deepfake detection increasingly challenging, exposing the limitations of current detection techniques. In this paper, we present HOLA, our solution to the Video-Level Deepfake Detection track of 2025 1M-Deepfakes Detection Challenge. Inspired by the success of large-scale pre-training in the general domain, we first scale audio-visual self-supervised pre-training in the multimodal video-level deepfake detection, which leverages our self-built dataset of 1.81M samples, thereby leading to a unified two-stage framework. To be specific, HOLA features an iterative-aware cross-modal learning module for selective audio-visual interactions, hierarchical contextual modeling with gated aggregations under the local-global perspective, and a pyramid-like refiner for scale-aware cross-grained semantic enhancements. Moreover, we propose the pseudo supervised singal injection strategy to further boost model performance. Extensive experiments across expert models and MLLMs impressivly demonstrate the effectiveness of our proposed HOLA. We also conduct a series of ablation studies to explore the crucial design factors of our introduced components. Remarkably, our HOLA ranks 1st, outperforming the second by 0.0476 AUC on the TestA set.
The rapid surge of text-to-speech and face-voice reenactment models makes video fabrication easier and highly realistic. To encounter this problem, we require datasets that rich in type of generation methods and perturbation strategy which is usually common for online videos. To this end, we propose AV-Deepfake1M++, an extension of the AV-Deepfake1M having 2 million video clips with diversified manipulation strategy and audio-visual perturbation. This paper includes the description of data generation strategies along with benchmarking of AV-Deepfake1M++ using state-of-the-art methods. We believe that this dataset will play a pivotal role in facilitating research in Deepfake domain. Based on this dataset, we host the 2025 1M-Deepfakes Detection Challenge. The challenge details, dataset and evaluation scripts are available online under a research-only license at https://deepfakes1m.github.io/2025.
Reproducibility is indispensable for transferring explainable-AI algorithms from academic prototypes to production systems. This companion paper documents the artefacts, procedures, and outcomes that reproduce the empirical claims of ''Enhancing Model Interpretability with Local Attribution over Global Exploration'' (ACM MM 2024). We release a containerised archive containing source code, data-serialisation scripts, one-click executables, and a detailed README, all conforming to the ACM Multimedia reproducibility guidelines. The regenerated Insertion and Deletion scores deviate by only 2.2% on average. In addition, an exhaustive 10, 20, 30 3 grid-search over key hyper-parameters reveals a new configuration, (30, 20, 30), that improves the Insertion score of three convolutional backbones by 7.51% without additional code changes. These artefacts provide a rigorous, extensible foundation for future research on local attribution methods. Our code is available at: https://github.com/LMBTough/LA/
This companion paper provides artifacts and instructions on replicating the experiments in the ACM Multimedia 2024 paper entitled ''Swarical: An Integrated Hierarchical Approach to Localizing Flying Light Specks.'' Swarm-based hierarchical, Swarical, is a localization technique that enables miniature drones, Flying Light Specks (FLSs), to accurately and efficiently localize and illuminate complex 2D and 3D shapes. It consists of two components, an offline planner and an online localization technique that executes on an FLS. The offline planner uses the FLS sensor specification for positioning to convert mesh files into swarms of FLSs. Some FLSs are dark and used only for localization. We reported the online localization technique to be fast and highly accurate. We describe how to reproduce this finding using our artifacts.