Illustration Layout Generation for Slide Enhancement with Pixel-based Diffusion Model

摘要

Embellishing slides with illustrations is a well-established practice for improving engagement and storytelling. However, this process is challenging, requiring careful consideration of both visual appearance and semantics of illustrations while ensuring they complement rather than overwhelm the slide content. In this paper, we take a pioneering step toward automating this process by introducing the task of Illustration Layout Generation: given a slide and a set of illustrations, automatically determining their optimal sizes and positions to enrich the slide. Existing layout generation approaches struggle with this task as they rely on large-scale layout datasets for training and have limited support for multiple visual inputs. To address these challenges, we propose SlideILG, a method that iteratively optimizes illustration placement using a diffusion-based text-to-image prior. We introduce three key techniques to enhance efficiency and quality: (1) leveraging cross-attention maps from the text-to-image model to initialize illustration placement; (2) employing an over-parameterization strategy to stabilize optimization; and (3) fine-tuning the text-to-image model on high-quality slide thumbnails for more precise guidance. To evaluate SlideILG, we construct IllustrationBench, a benchmark comprising 128 real-world slides, each paired with a set of illustrations for embellishment. Quantitative, qualitative and human-study results demonstrate the effectiveness of our approach. Furthermore, we showcase a real-world application scenario to highlight the significance and practical utility of this task and our method.