AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs
Fudan University · Huazhong University of Science and Technology · Shanghai Jiaotong University · Renmin University of China · Nanyang Technological University · Tsinghua University · University of Edinburgh · University of Oxford · Institute for AI Industry Research, Tsinghua University · Institute for AI Industry Research (AIR), Tsinghua University
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Recent advances in Omni-Multimodal Large Language Models (Omni-MLLMs) have enabled strong integration of vision, audio, and language. However, their audio-visual intelligence (AVI) remains insufficiently evaluated due to the lack of systematic and comprehensive benchmarks. We introduce AVI-Bench, a cognitively inspired benchmark that evaluates Omni-MLLMs across three stages, perception, understanding, and reasoning, through cross-modal tasks requiring joint audio-visual interpretation. This design enables fine-grained diagnosis of model capabilities and failure modes. To further assess robustness beyond familiar domains, we propose AVI-Bench-PriSe, an extension that probes models' primitive audio-visual sensation using unfamiliar, low-semantic stimuli, testing generalization beyond common training distributions. Extensive experiments on both open-source and closed-source models reveal substantial limitations in current Omni-MLLMs. Based on these findings, we present a four-level AVI taxonomy. Overall, AVI-Bench provides a principled evaluation framework to guide the development of more robust and generalizable AVI.