LargePiG for Hallucination-Free Query Generation: Your Large Language Model is Secretly a Pointer Generator

摘要

Recent research on query generation has focused on using Large Language Models (LLMs), which, despite achieving state-of-the-art performance, also introduce hallucination issues in generated queries. In this work, we categorize these issues into relevance hallucination and factuality hallucination, proposing a new typology for hallucinations arising from LLM-based query generation. We present an effective approach to decouple content from form in LLM-generated queries, preserving the factual knowledge extracted and integrated from inputs while leveraging the LLM's linguistic capabilities to construct syntactic structures, including function words. Specifically, we introduce a model-agnostic and training-free method that transforms the Large Language Model into a Pointer-Generator (LargePiG), where the pointer attention distribution utilizes the LLM's inherent attention weights, and the copy probability is derived from the difference between the vocabulary distribution in the model's high layers and the last layer. To validate the effectiveness of LargePiG, we constructed two datasets for assessing hallucination issues in query generation, covering both document and video scenarios. Empirical studies on various LLMs demonstrated LargePiG's superiority across both datasets. Additional experiments further verified that LargePiG reduces hallucination in large vision-language models and enhances the accuracy of document-based question-answering and factuality evaluation tasks. The source code and dataset are available at https://github.com/Jeryi-Sun/LargePiG.