DEPO: Enhancing E-commerce Image Background Generation with Short Trajectory Direct Expected Preference Optimization

摘要

Generating high-quality, user-preferred backgrounds for e-commerce product images poses unique challenges for diffusion models, particularly in aligning outputs with human visual preferences. While Direct Preference Optimization (DPO) has shown promise in aligning generative models with human feedback, its application to diffusion models faces key limitations, including the trade-off between reward sparsity and supervision quality, mode collapse, and training instability. To tackle these issues, we propose Direct Expected Preference Optimization (DEPO), a novel framework that adapts DPO to diffusion models through redesigned training and sampling strategies. Specifically, DEPO introduces a DEPO loss combined with trajectory segmentation to enable more frequent and informative reward feedback, employs Langevin MCMC to broaden the exploration space and mitigate mode collapse, and leverages masks to effectively constrain the search space while incorporating targeted engineering designs to improve training stability. By directly linking image-domain evaluations to expected log probabilities and incorporating adversarial training, DEPO achieves better alignment with user preferences while maintaining high image fidelity. Experimental results demonstrate that DEPO surpasses existing methods in both the diversity and quality of background generation.