Rethinking the Reliability of Evidence in End-to-End Fact-Checking from the Causal Perspective

摘要

End-to-end automated fact-checking (AFC) aims to assess the truthfulness of claims using retrieved evidence. Some researchers use crawlers or search APIs to retrieve evidence from the web for veracity classification. However, existing methods indiscriminately rely on the retrieved evidence and overlook that the retrieved results are not always reliable. This unilateral reliance on evidence significantly hampers the performance of fact-checking. In this paper, we account for the diverse reliability levels of retrieved evidence and eliminate the negative impact from the causal perspective. To achieve our goal, we propose a novel Causal intervention and Counterfactual reasoning based Multi-Checker framework (CCMC), which introduces two additional counterfactual fact-checkers to verify claims from the counterfactual perspective. Specifically, we construct two distinct types of counterfactual instances via causal intervention to imitate the situation where the evidence is partially reliable or totally unreliable. Correspondingly, two counterfactual fact-checkers are trained with tailored counterfactual instances by counterfactual reasoning. During inference, the two counterfactual fact-checkers are employed to estimate and eliminate the potential impact of unreliable evidence. Extensive experiments on two real-world datasets demonstrate the superiority of our approach for improving end-to-end AFC. Especially, we surpass existing methods by 3.70% and 5.55% under gold and system evidence on the MOCHEG benchmark, respectively. Our code is available at https://github.com/BeiyuXuboL/CCMC.