ENRIC: EveNt-AwaRe Captioning with Image Retrieval via UnCertainty-Guided Re-ranking and Semantic Ensemble Reasoning

摘要

In many real-world applications, labeling an image ''a man riding a horse'' fails to satisfy demands for the who, when, where, and why. Although LVLMs excel at describing visual content, isolated images often lack the event context; users thus rely on related news articles or social posts to enrich them, but cropping or resizing complicates tracking back to their source. In this paper, we propose ENRIC, an innovative end-to-end system for the EVENTA Challenge Track 1, leveraging the OpenEvents-V1 dataset, comprising over 200,000 news articles paired with more than 400,000 images. Our system includes three components: (1) semantic retrieval filters candidate article images via vision-language embeddings, (2) uncertainty-guided re-ranking flags ambiguous queries using three confidence heuristics and re-ranks candidates by combining visual similarity with texture similarity, and (3) event-aware caption generation employs chain-of-thought prompting that aggregates five inputs from article, image, and CIDEr-derived contexts to guide the LLM in incorporating all necessary elements. ENRIC achieved the highest combined evaluation score of 0.5501, ranking first and outperforming other solutions across nearly all metrics. By combining semantic retrieval, uncertainty-guided re-ranking, and event-aware caption generation, ENRIC demonstrates the efficiency of its approach for event-enriched image analysis. GitHub repository: https://github.com/NamQuanProject/EVENTA25-ENRIC