Free-Mask: A Novel Paradigm of Integration Between the Segmentation Diffusion Model and Image Editing

摘要

Current semantic segmentation models typically require a substantial amount of manually annotated data, a process that is both time-consuming and resource-intensive. Alternatively, leveraging advanced text-to-image models such as Midjourney and Stable Diffusion has emerged as an efficient strategy, enabling the automatic generation of synthetic data in place of manual annotations. However, previous methods have been limited to generating single-instance images, as the generation of multiple instances with Stable Diffusion has proven unstable and masks can be significantly affected by occlusion between different objects. To overcome this limitation and broaden the variety of synthetic datasets, we propose a novel framework, Free-Mask. It combines a Diffusion Model for segmentation with advanced image editing capabilities, allowing the insertion of multiple objects into images through text-to-image models. In addition, we introduce a new active learning paradigm that benefits both model generalization and data optimization. Our method enables the creation of realistic datasets that closely reflect open-world environments while generating accurate segmentation masks. Our code is released on GitHub.