Propaganda Signals in LLMs: Perspectival Divergence and Narrative Framing in the Russia-Ukraine War

摘要

Large Language Models (LLMs) are increasingly used to explain, summarize, and translate real-world events, including ongoing geopolitical conflicts. Yet it remains unclear whether they reproduce conflict-specific propaganda and, if so, how this appears in their outputs. We study this question for the Russia-Ukraine war through perspectival divergence, the extent to which model outputs align with competing narratives from different information ecosystems. We construct a conflict-aware evaluation set of neutral English event statements paired with Russian (RU)- and Ukrainian (UA)-oriented reference texts drawn from news outlets and Telegram channels. We then evaluate multiple LLMs under several prompting contexts using a reference-based semantic distance metric that measures directional proximity to RU- and UA-oriented references. To explain not only which side a model is closer to but also how that alignment is expressed, we further analyze outputs using five propaganda-relevant categories: Framing Narrative, Emotional Manipulation, Source Credibility, Social Pressure Identity, and Toponymy Naming. Across models, we find stable, model-specific leanings and technique profiles that persist across prompts and are not captured by standard factuality-oriented metrics. Our findings show that models that appear neutral under conventional evaluations can still encode systematic, conflict-specific propaganda patterns, underscoring the need for conflict-aware evaluation frameworks.