Retrieval Augmented 3D Garment Generation from Single Image

摘要

In this work, We propose a novel framework for 3D garment generation, named Retrieval Augmented 3D Garment Generation (RAG2), capable of generating high-quality mesh and high consistent texture with input image simultaneously. Specifically, we decouple the 3D garment generation task into garment modeling and texturing to address the issues of low-quality meshes and poor texture consistency caused by using a single model in previous approaches. For garment modeling, we build a base garment mesh database and introduce Retrieval Augmented Deformer to obtain high-quality mesh with similar clothing styles to the input image. To generate high-fidelity texture, we propose TextureNet by imposing a high-fidelity UV generation module to ensure consistency with the input image; a multi-view consistent branch to ensure geometry and logical coherence; and a DiT-based main branch to support efficient and dedicated information interaction between multi-branches. Extensive experiments validate that RAG2 surpasses existing methods both in mesh quality and texture fidelity.