Garment sewing patterns are the design language behind clothing, yet their current vector-based digital representations weren’t built with machine learning in mind. Vector-based representation encodes a sewing pattern as a discrete set of panels, each defined as a sequence of lines and curves, stitching information between panels and the placement of each panel around a body. However, this representation causes two major challenges for neural networks: discontinuity in latent space between patterns with different topologies and limited generalization to garments with unseen topologies in the training data. In this work, we introduce GarmentImage, a unified raster-based sewing pattern representation. GarmentImage encodes a garment sewing pattern’s geometry, topology and placement into multi-channel regular grids. Machine learning models trained on GarmentImage achieve seamless transitions between patterns with different topologies and show better generalization capabilities compared to models trained on vector-based representation. We demonstrate the effectiveness of GarmentImage across three applications: pattern exploration in latent space, text-based pattern editing, and image-to-pattern prediction. The results show that GarmentImage achieves superior performance on these applications using only simple convolutional networks.
论文检索
输入标题、作者或关键词，从 306 篇学术成果中精准定位
We present the first algorithm to automatically compute sewing patterns for upcycling existing garments into new designs. Our algorithm takes as input two garment designs along with their corresponding sewing patterns and determines how to cut one of them to match the other by following garment reuse principles. Specifically, our algorithm favors the reuse of seams and hems present in the existing garment, thereby preserving the embedded value of these structural components and simplifying the fabrication of the new garment. Finding optimal reused pattern is computationally challenging because it involves both discrete and continuous quantities. Discrete decisions include the choice of existing panels to cut from and the choice of seams and hems to reuse. Continuous variables include the precise placement of the new panels along seams and hems, and potential deformations of these panels to maximize reuse. Our key idea for making this optimization tractable is quantizing the shape of garment panels. This allows us to frame the search for an optimal reused pattern as a discrete assignment problem, which we solve efficiently with an ILP solver. We showcase our proposed pipeline on several reuse examples, including comparisons with reused patterns crafted by a professional garment designer. Additionally, we manufacture a physical reused garment to demonstrate the practical effectiveness of our approach.
Manual design of garments for avatars requires a large effort. Garment retargeting methods can save manual efforts by automatically deforming an existing garment design from one avatar to another. Previous methods are limited to human avatars with small variations in body shapes, while non-human avatars with unrealistic characteristics widely appear in games and animations. In this paper, the goal is to retarget artist-designed garments on a standard mannequin to a more general class of avatars. While there is a lack of training data of various avatars wearing garments, we propose a training-free method that performs optimizations on the mesh representation of the garments, with a combination of loss functions that preserve the geometrical features in the original design, guarantee intersection-free, and fit the garment adaptively to the avatars. Our method produces simulation-ready garment models that can be used later in avatar animations.
Knots and ties are captivating elements of digital garments and accessories, but they have been notoriously challenging and computationally expensive to model manually. In this paper, we propose a physics-based modeling system for knots and ties using templates. The primary challenge lies in transforming cloth pieces into desired knot and tie configurations in a controllable, penetration-free manner, particularly when interacting with surrounding meshes. To address this, we introduce a pipe-like parametric knot template representation, defined by a Bézier curve as its medial axis and an adaptively adjustable radius for enhanced flexibility and variation. This representation enables customizable knot sizes, shapes, and styles while ensuring intersection-free results through robust collision detection techniques. Using the defined knot template, we present a mapping and penetration-free initialization method to transform selected cloth regions from UV space into the initial 3D knot shape. We further enable quasistatic simulation of knots and their surrounding meshes through a fast and reliable collision handling and simulation scheme. Our experiments demonstrate the system’s effectiveness and efficiency in modeling a wide range of digital knots and ties with diverse styles and shapes, including configurations that were previously impractical to create manually.
The true vision for real-time virtual and augmented reality is reproducing our visual reality in its entirety on immersive displays. To this end, foveated rendering leverages the limitations of spatial acuity in human peripheral vision to allocate computational resources to the fovea while reducing quality in the periphery. Such methods are often derived from studies on the spatial resolution of the human visual system and its ability to perceive blur in the periphery, enabling the potential for high spatial quality in real-time. However, the effects of blur on other visual cues that depend on luminance contrast, such as depth, remain largely unexplored. It is critical to understand this interplay, as accurate depth representation is a fundamental aspect of visual realism. In this paper, we present the first evaluation exploring the effects of foveated rendering on stereoscopic depth perception. We design a psychovisual experiment to quantitatively study the effects of peripheral blur on depth perception. Our analysis demonstrates that stereoscopic acuity remains unaffected (or even improves) by high levels of peripheral blur. Based on our studies, we derive a simple perceptual model that determines the amount of foveation that does not affect stereoacuity. Furthermore, we analyze the model in the context of common foveation practices reported in literature. The findings indicate that foveated rendering does not impact stereoscopic depth perception, and stereoacuity remains unaffected with up to 2 × stronger foveation than commonly used. Finally, we conduct a validation experiment and show that our findings hold for complex natural stimuli.
Facial expressions are key to human communication, conveying emotions and intentions. Given the rising popularity of digital humans and avatars, the ability to accurately represent facial expressions in real time has become an important topic. However, quantifying perceived differences between pairs of expressions is difficult, and no comprehensive subjective datasets are available for testing. This work introduces a new dataset targeting this problem: FaceExpressions-70k. Obtained via crowdsourcing, our dataset contains 70,500 subjective expression comparisons rated by over 1,000 study participants1 We demonstrate the applicability of the dataset for training perceptual expression difference models and guiding decisions on acceptable latency and sampling rates for facial expressions when driving a face avatar.
The contrast and luminance capabilities of a display are central to the quality of the image. High dynamic range (HDR) displays have high luminance and contrast, but it can be difficult to ascertain whether a given set of characteristics qualifies for this label. This is especially unclear for new display modes, such as virtual reality (VR). This paper studies the perceptual impact of peak luminance and contrast of a display, including characteristics and use cases representative of VR. To achieve this goal, we first developed a haploscope testbed prototype display capable of achieving 1,000 nits peak luminance and 1,000,000:1 contrast with high precision. We then collected a novel HDR video dataset targetting VR-relevant content types. We also implemented custom tone mapping operators to map between display parameter sets. Finally, we collected subjective preference data spanning 3 orders of magnitude in each dimension. Our data was used to fit a model, which was validated using a subjective study on an HDR VR prototype headmounted display (HMD). Our model helps provide guidance for future display design, and helps standardize the understanding of HDR1.
We explore interactive painting on 3D Gaussian splat scenes and other surfaces using 3D Gaussian splat brushes, each containing a chunk of realistic texture-geometry that make capture representations so appealing. The suite of brush capabilities we propose enables 3D artists to capture and then remix real world imagery and geometry with direct interactive control. In particular, we propose a set of algorithms for 1) selecting subsets of Gaussians as a brush pattern interactively, 2) applying the brush interactively to the same or other 3DGS scenes or other 3D surfaces using stamp-based painting, 3) using an inpainting Diffusion Model to adjust stamp seams for seamless and realistic appearance. We also present an ensemble of artistic brush parameters, resulting in a wide range of appearance options for the same brush. Our contribution is a judicious combination of algorithms, design features and creative affordances, that together enable the first prototype implementation of interactive brush-based painting with 3D Gaussian splats. We evaluate our system by showing compelling results on a diverse set of 3D scenes; and a user study with VFX/animation professionals, to validate system features, workflow, and potential for creative impact. Code and data for this paper can be accessed from splatpainting.github.io.
Previous research in material models for surface and volume scattering has enabled highly realistic scenes in modern rendering systems. However, there has been comparatively little study of light sources in computer graphics despite their critical importance in illuminating and bringing life into these scenes. In the real world, photons are emitted through numerous physical processes including combustion, incandescence, and fluorescence. The qualities of light produced in each of these processes are unique to their physics, making them interesting to study individually. In this work, we propose a model for glow discharge, a form of light-emitting electrostatic discharge commonly found in Neon lights and gas discharge lamps. We take inspiration from works in computational physics and develop an efficient point-wise solver for the emission due to glow discharge suitable for traditional volume rendering systems. Our model distills the complex mechanics of this process into a set of flexible and interpretable parameters. We demonstrate that our model can replicate the visual qualities of glow discharge under varying gases.
We present a method for generating stroke-based painterly drawings of participating media, such as smoke, fire, and clouds, by transferring stroke attributes—color, width, length, and orientation—from exemplar to animation frames. Building on the stroke transfer framework, we introduce features and basis fields capturing lighting-, view-, and geometry-dependent information, extending surface-based ones (e.g., intensity, apparent normals and curvatures, and distance from silhouettes) to volumetric scenes while supporting traditional surface objects. Novel features, including apparent relative velocity and mean free-path, address non-rigid flow and dynamic scenes. Our system combines automated exemplar selection, user-guided style learning, and temporally coherent stroke generation, enabling artistic and expressive visualizations of dynamic media.
Existing neural shadow mapping methods [Datta et al. 2022] have shown to be promising in generating high quality soft shadows. However, it demonstrates limited generalizability to new scenes. In this paper, we present a novel neural method, named kernel predicting neural shadow mapping to address this issue. Specifically, we explicitly model soft shadow values as pixelwise local filtering from nearby base shadow values (i.e., the classic hard shadow values) in the screen space, where the local filter weights are predicted through a trained neural network. We use dilated filters as the representation of our local filters to maintain a balance between computational efficiency and receptive field of a local filter. We further enhance shadow quality by replacing the classic shadow map algorithm [Williams 1978] with moment shadow maps [Peters and Klein 2015] to generate the base shadows values. With carefully designed filters, input features, and loss functions with temporal regularization, our method runs in real-time framerates (i.e., >100 fps for 2048 × 1024 resolution), produces temporally-stable soft shadows with good generalizability, and consistently beats state-of-the-art methods in both visual qualities and numeric measures. Code and model weights are available at https://github.com/Hoosus/KPNSM.
Recent extensions to spatiotemporal path reuse, or ReSTIR, improve rendering efficiency in the presence of high-frequency content by augmenting path reservoirs to represent contributions over full pixel footprints. Still, if historical paths fail to contribute to future frames, these benefits disappear. Prior ReSTIR work backprojects to the prior frame to identify paths for reuse. Backprojection can fail to find relevant paths for many reasons, including moving cameras or subpixel geometry with differing motion. We introduce reservoir splatting to reduce these failures. Splatting forward-projects the primary hits of prior-frame paths. Unlike backprojection, forward-projected path samples fall into the current-frame pixel relevant to their exact primary hits, making successful reuse more likely. This also enables motion blur for ReSTIR, by splatting at multiple time steps, and supports depth of field without the specialized shift maps needed previously. Beyond enabling motion blur, splatting improves resampling quality over Zhang et al.’s [2024] Area ReSTIR at up to 10% lower cost. To improve robustness, we show how to MIS splatted and backprojected samples to help every current-frame pixel get at least one historical path proposed for reuse.
We introduce a guided lens sampling method for efficient rendering of circles of confusion (CoCs). While traditional Monte Carlo techniques simulate depth-of-field (DoF) effects by perturbing camera rays at the lens, uniform lens sampling often results in significant noise by failing to prioritize rays toward highlight regions in the scene. Although path guiding has proven effective for global illumination by learning importance distributions for incoming radiance, no comparable guiding technique for CoCs exists, primarily due to the strong parallax between adjacent pixels. We model highlight spots in world space using a globally shared radiance field, which is then transformed into lens space through a bipolar-cone projection to guide camera ray generation. We implement this theory using 3D Gaussians, achieving fast, robust guiding with minimal computational and storage overhead, making it suitable for production rendering. We also propose two extensions to further enhance local adaptation. Our experiments show that this approach significantly improves the sampling efficiency for CoC rendering.
In this paper, we introduce a second-order derivative of spherical harmonics, spherical harmonics Hessian, and solid spherical harmonics, a variant of spherical harmonics, to the computer graphics community. These mathematical tools are used to develop an analytical representation of the Hessian matrix of spherical harmonics coefficients for spherical lights. We apply our analytic representation of the Hessian matrix to grid-based SH lighting rendering applications with many spherical lights that store the incident light field as spherical harmonics coefficients and their spatial gradient at sparse grid. We develop a Hessian-based error metric, with which our method automatically and adaptively subdivides the grid whether the interpolation using the spatial gradient is appropriate. Our method can be easily incorporated into the grid-based precomputed radiance transfer (PRT) framework with small additional storage. We demonstrate that our adaptive grid subdivided by using the Hessian-based error metric can substantially improve the rendering quality in equal-time grid construction.
Autoregressive models have achieved remarkable success across various domains, yet their performance in 3D shape generation lags significantly behind that of diffusion models. In this paper, we introduce OctGPT, a novel multiscale autoregressive model for 3D shape generation that dramatically improves the efficiency and performance of prior 3D autoregressive approaches, while rivaling or surpassing state-of-the-art diffusion models. Our method employs a serialized octree representation to efficiently capture the hierarchical and spatial structures of 3D shapes. Coarse geometry is encoded via octree structures, while fine-grained details are represented by binary tokens generated using a vector quantized variational autoencoder (VQVAE), transforming 3D shapes into compact multiscale binary sequences suitable for autoregressive prediction. To address the computational challenges of handling long sequences, we incorporate octree-based transformers enhanced with 3D rotary positional encodings, scale-specific embeddings, and token-parallel generation schemes. These innovations reduce training time by 13 folds and generation time by 69 folds, enabling the efficient training of high-resolution 3D shapes, e.g.,10243, on just four NVIDIA 4090 GPUs only within days. OctGPT showcases exceptional versatility across various tasks, including text-, sketch-, and image-conditioned generation, as well as scene-level synthesis involving multiple objects. Extensive experiments demonstrate that OctGPT accelerates convergence and improves generation quality over prior autoregressive methods, offering a new paradigm for high-quality, scalable 3D content creation. Our code and trained models are available at https://github.com/octree-nn/octgpt.
Transforming 3D shapes into representations that support part-level editing, flexible redesign, and efficient compression is vital for asset customization, content creation, and optimization in digital design. Despite its importance, achieving a representation that balances expressivity, editability, compactness, and interpretability remains a challenge. We introduce 3D2EP, a novel method for 3D shape decomposition that represents objects as a collection of differentiable, parametric primitives. Given a 3D shape represented by a voxel grid, 3D2EP decomposes this into a set of primitive parts, each generated by extruding a scaled 2D profile along a 3D curve, with the requisite components being predicted in a feedforward manner. That is, each primitive is constrained to have a single cross-section profile, up to scale. This enables the primitives to adapt to the data, capturing the geometry with precision but without excess degrees-of-freedom that would stymie editability. Extensive evaluations highlight 3D2EP’s ability to reconstruct complex shapes with a compact and interpretable representation, emphasizing its suitability for a wide range of 3D modeling applications.
We introduce Masked Anchored SpHerical Distances (MASH), a novel multi-view and parametrized representation of 3D shapes. Inspired by multi-view geometry and motivated by the importance of perceptual shape understanding for learning 3D shapes, MASH represents a 3D shape as a collection of observable local surface patches, each defined by a spherical distance function emanating from an anchor point. We further leverage the compactness of spherical harmonics to encode the MASH functions, combined with a generalized view cone with a parameterized base that masks the spatial extent of the spherical function to attain locality. We develop a differentiable optimization algorithm capable of converting any point cloud into a MASH representation accurately approximating ground-truth surfaces with arbitrary geometry and topology. Extensive experiments demonstrate that MASH is versatile for multiple applications including surface reconstruction, shape generation, completion, and blending, achieving superior performance thanks to its unique representation encompassing both implicit and explicit features. More information and resources can be found at: https://chli.top/MASH.
Obtaining 3D faces with microscopic structures from a single unconstrained image is challenging. The complexities of wrinkles and pores at a microscopic level, coupled with the blurriness of the input image, raise the difficulty. However, the distribution of wrinkles and pores tends to follow a specialized pattern, which can provide a strong prior for synthesizing them. Therefore, a key to microstructure synthesis is a parametric wrinkles and pore model with controllable semantic parameters. Additionally, ensuring differentiability is essential for enabling optimization through gradient descent methods. To this end, we propose a novel framework designed to reconstruct facial micro-wrinkles and pores from naturally captured images efficiently. At the core of our framework is a differentiable representation of wrinkles and pores via a graph neural network (GNN), which can simulate the complex interactions between adjacent wrinkles by multiple graph convolutions. Furthermore, to overcome the problem of inconsistency between the blurry input and clear wrinkles during optimization, we proposed a Direction Distribution Similarity that ensures that the wrinkle-directional features remain consistent. Consequently, our framework can synthesize facial micro-structures from a blurry skin image patch, which is cropped from a natural-captured facial image, in around an average of 2 seconds. Our framework can seamlessly integrate with existing macroscopic facial detail reconstruction methods to enhance their detailed appearance. We showcase this capability on several works, including DECA, HRN, and FaceScape.
Creating animatable 3D avatars from a single image remains challenging due to style limitations (realistic, cartoon, anime) and difficulties in handling accessories or hairstyles. While 3D diffusion models advance single-view reconstruction for general objects, outputs often lack animation controls or suffer from artifacts because of the domain gap. We propose SOAP, a style-omniscient framework to generate rigged, topology-consistent avatars from any portrait. Our method leverages a multiview diffusion model trained on 24K 3D heads with multiple styles and an adaptive optimization pipeline to deform the FLAME mesh while maintaining topology and rigging via differentiable rendering. The resulting textured avatars support FACS-based animation, integrate with eyeballs and teeth, and preserve details like braided hair or accessories. Extensive experiments demonstrate the superiority of our method over state-of-the-art techniques for both single-view head modeling and diffusion-based generation of Image-to-3D. Our code and data are publicly available for research purposes at github.com/TingtingLiao/soap.
We present LAM, an innovative Large Avatar Model for animatable Gaussian head reconstruction from a single image. Unlike previous methods that require extensive training on captured video sequences or rely on auxiliary neural networks for animation and rendering during inference, our approach generates Gaussian heads that are immediately animatable and renderable. Specifically, LAM creates an animatable Gaussian head in a single forward pass in seconds, enabling reenactment and rendering without additional networks or post-processing steps. This capability allows for seamless integration into existing rendering pipelines, ensuring real-time animation and rendering across a wide range of platforms, including mobile phones. The centerpiece of our framework is the canonical Gaussian attributes generator, which utilizes FLAME canonical points as queries. These points interact with multi-scale image features through a Transformer to accurately predict Gaussian attributes in the canonical space. The reconstructed canonical Gaussian avatar can then be animated utilizing standard linear blend skinning (LBS) with corrective blendshapes as the FLAME model did and rendered in real-time on various platforms. The experiments demonstrate that LAM outperforms state-of-the-art methods on existing benchmarks.