SAKR-Edit: Scene-Aware Knowledge Reasoning for Text-to-Image Editing

摘要

Image editing requires semantically modifying specific regions according to user instructions while preserving overall visual coherence. Although diffusion models have shown remarkable progress in image generation, their application to editing tasks faces two critical limitations: (1) insufficient understanding of editing objectives often leads to inconsistencies in style, attribute, or texture between generated content and background regions, and (2) over-reliance on ambiguous textual prompts that frequently lack crucial details, resulting in suboptimal edits. To address these challenges, we propose SAKR-Edit, a novel framework that enhances editing quality and controllability through Scene-Aware Knowledge Reasoning. Specifically, our approach introduces a scene-aware knowledge reasoning module that combines large language models (LLMs) with vision-language models (e.g., BLIP-2) to integrate global and local semantic information for improved instruction comprehension. The system employs chain-of-thought reasoning and contextual learning to parse instructions, infer implicit editing intentions, and supplement missing details, thereby improving editing precision. Additionally, we construct SSUD, a structured scene understanding dataset for evaluating editing models in real-world scenarios. Extensive experiments demonstrate that SAKR-Edit outperforms existing methods in image realism, style consistency, and structural integrity, while showing robust stability and adaptability in real-world applications. Our code and dataset are released at https://github.com/SAKR-Edit/sakr-edit.github.io.