HEAR: A Holistic Extraction and Agentic Reasoning Framework for Document Understanding

摘要

The automated comprehension of complex, multi-modal documents is fundamentally hampered by a disconnect between information extraction and reasoning. Existing systems suffer from inherent limitations. Monolithic models embed reasoning as a black box process, sacrificing transparency and depth. Meanwhile, current agent-based frameworks follow a passive, non-interactive paradigm; they handle static, global inputs rather than information derived from active exploration, which fundamentally restricts their ability to achieve structural understanding and complex reasoning. To bridge this critical gap, we introduce HEAR, a framework for Holistic Extraction and Agentic Reasoning. This innovative framework establishes a synergistic, closed-loop between a deep Vision-Language Model (VLM) driven holistic parsing engine and a collaborative multi-agent reasoning system. Our HEAR initially transforms unstructured documents into a semantically-rich, structured representation, preserving complex layouts and reconstituting multi-page tables. Subsequently, a multi-agent system performs cross-modal analysis, governed by a crucial verification protocol that forces agents to validate findings across textual and visual modalities. A conflict driven re-evaluation mechanism enables the system to dynamically re-engage the document to resolve ambiguities, thereby unifying the perception-cognition cycle. HEAR achieved first place in the ACM MM 2025 Grand Challenge on Large Vision–Language Model Learning and Applications.