AHEAD: Attention Head Energy-Aware Dynamics for Hallucination Mitigation in MLLMs

摘要

Multimodal large language models excel at vision-language tasks but remain prone to hallucinations that undermine their reliability. Existing approaches predominantly treat hallucinations as classification errors, overlooking the heterogeneous behaviors of attention heads and their dynamic influences during inference. We revisit MLLM reasoning from an energy perspective and identify that hallucinations stem from imbalances between visual potential and language prior potential: when visual information is ambiguous or language priors dominate, attention heads tend to be driven by linguistic statistical patterns, generating content inconsistent with visual evidence. We propose AHEAD, a framework that quantifies the energetic properties of each attention head during object generation through two potential networks—the Visual Grounding Potential Network and the Language Prior Potential Network—and dynamically adjusts their contributions at inference time. Specifically, we amplify attention heads with strong visual grounding capacity while suppressing those overly reliant on language priors. Experiments across multiple benchmarks demonstrate that AHEAD significantly reduces hallucination rates without fine-tuning the base MLLM while maintaining generation quality.