Two-Stage Approach Using Pretrained Language Models for Question Answering on Japanese Document Images

摘要

LAVA Challenge 2025 aims to improve the ability of large visual language models to accurately understand complex visual information such as data flow diagrams and Gantt charts contained in Japanese government and business documents. For this challenge, we adopted a two-stage approach consisting of retrieval and reading comprehension. Specifically, in the retrieval step, we select pages relevant to the question from multi-page PDF documents, and in the reading comprehension step, we perform question answering by referring to the top k images selected in the retrieval step. For the retrieval step, we employ ColQwen2, which performs visual information retrieval using the multilingual Qwen2-VL. For the reading comprehension step, we propose a method that performs question answering, including voting, using the multilingual visual language model Qwen2.5VL under different prompts, model sizes, and image qualities. In LAVA Challenge 2025, we clarify the importance of the two-step process of retrieval search and reading comprehension in visual question answering, and verify the effectiveness of a method for determining the multiple results of reading comprehension steps through voting.