Capturing cloth motion with high fidelity is challenging due to fine-scale wrinkles, large deformation, and frequent self-occlusion. We present a 4D (spatio-temporal) cloth capture system that achieves 1 mm spatial resolution using only 16 RGB cameras. Our approach uses a two-level marker pattern: sparse, colored L-shaped markers provide robust detection and orientation, while dense noise patterns within each marker enable both marker identification and precise keypoint localization. By unwarping detected markers to a canonical frame, we factor out perspective distortion and most of cloth deformation, allowing the localizer to achieve sub-pixel accuracy. The localized keypoints are triangulated across views to form an incomplete point cloud. A physics-based optimization then deforms a template mesh to match the captured geometry while maintaining penetration-free constraints and physical plausibility for occluded regions. Our method produces temporally coherent sequences that faithfully capture fine wrinkles and folds even during complex motions with self-contact.
论文检索
输入标题、作者或关键词，从 17,487 篇学术成果中精准定位
The rapid adoption of volumetric capture technologies has created a pressing need for efficient storage and streaming of dynamic 3D content. Unfortunately, current compression standards often treat dynamic sequences as independent frames or rely on computationally expensive non-rigid registration, making them unsuitable for real-time applications or large-scale environments. In this paper, we present a novel end-to-end compression framework for dynamic Truncated Signed Distance Field volumes derived from captured 3D content, leveraging a representation that is temporally stable, easily parallelizable, and already widely used in scene reconstruction and volumetric fusion pipelines. We then adapt classic 2D video coding paradigms such as spatial coding via Discrete Cosine Transform and temporal coding using a real-time motion compensation pipeline to provide robust, real-time, and training-free encoding and decoding for 3D content. Extensive evaluations on human performance captures demonstrate that our codec achieves ~35% bitrate savings at equal distortion while operating in real time at 30 FPS, while stronger temporal coherence in large-scale synthetic environments yields up to 12× bitrate reduction at equal distortion.
This paper describes a strategy for vectorizing 3D scenes with proper occlusion. Given a collection of curves derived from 3D geometry (silhouettes, isolines, material boundaries, etc.), we produce a 2D vector image that partitions the image plane into a planar map of solid shaded regions. The method is agnostic to surface representation, handling for instance curves obtained from polygonal, NURBS or subdivision surfaces. The output likewise supports curve segments of arbitrary parametric type. Our key observation is that the spatial hierarchy used to accelerate curve-curve intersections provides the fundamental representation of the planar map itself. This approach provides geometric flexibility, since general curve-curve intersection problems are replaced with simpler curve-line intersection. Simultaneously, it provides robustness since cells of the spatial hierarchy define a well-defined planar map, even in the presence of numerical errors. For instance, it automatically handles "curve soup" where segment endpoints are not explicitly connected in the input file. The method scales to a large number of primitives, and is output sensitive: it resolves intersections only up to a user-defined precision—while still providing topologically valid output. We evaluate the method on a collection of challenging tests, showing that it is both more robust and orders of magnitude more efficient than existing curve arrangement techniques, such as those found in CGAL.
While yarn-level garments offer rich dynamic details and compelling visual realism compared to triangle-based models, their wide adoption is hindered by the immense computational cost due to the presence of a large number of degrees of freedom (DOFs). This paper proposes a novel simulation framework designed to enhance the performance and stability for numerical simulation of nonlinear, non-convex, and high-resolution knitwear. Our method generalizes the Douglas-Rachford Splitting (DRS) scheme to resolve the non-convex coupling between stretching, shearing, bending, twisting, and contacting at each yarn thread. A key contribution is a nested decomposition strategy that decouples the non-convex variational energy into independent and convex sub-problems. Such convexification improves solver robustness and removes the necessity for frequent line searches. We provide a theoretically grounded strategy for metric selection for each sub-problem, derived from an analysis of the convergence guarantee of DRS. Consequently, our method achieves close-to-optimal convergence along the nonlinear iterations rather than relying on ad-hoc parameter tuning. The paper also clarifies a formal connection between our generalized DRS and the ADMM (Alternating Direction Method of Multipliers) framework, extending the applicability of our analysis to a broader set of constrained dynamics problems. Experimental results demonstrate that our method robustly handles complex knitwear simulation scenes with superior efficiency, stability, and physical fidelity compared to existing methods. With a matrix-free GPU parallelization, our method allows an interactive simulation rate of knitwear of multi-million DOFs.
We present an efficient B-spline finite element method (FEM) for cloth simulation. While higher-order FEM has long promised higher accuracy, its adoption in cloth simulators has been limited by its larger computational costs while generating results with similar visual quality. Our contribution is a full algorithmic pipeline that makes cloth simulation using quadratic B-spline surfaces faster than standard linear FEM in practice while consistently improving accuracy and visual fidelity. Using quadratic B-spline basis functions, we obtain a globally C1-continuous displacement field that supports consistent discretization of both membrane and bending energies, effectively reducing locking artifacts and mesh dependence common to linear elements. To close the performance gap, we introduce a reduced integration scheme that separately optimizes quadrature rules for membrane and bending energies, an accelerated Hessian assembly procedure tailored to the spline structure, and an optimized linear solver based on partial factorization. Together, these optimizations make high-order, smooth cloth simulation competitive at scale, yielding an average 2× speedup over linear FEM in our tests. Extensive experiments demonstrate improved accuracy, wrinkle detail, and robustness, including contact-rich scenarios, relative to linear FEM and recent higher-order approaches. Our method enables realistic wrinkling dynamics across a wide range of material parameters and supports practical garment animation, providing a new promising spatial discretization for high-quality cloth simulation.
Retargeting human kinematic reference motion onto a robot's morphology remains a formidable challenge. Existing methods often produce physical inconsistencies, such as foot sliding, self-collisions, or dynamically infeasible motions, which hinder downstream imitation learning. We propose a bilevel optimization framework that jointly adapts reference motions to a robot's morphology while training a tracking policy using reinforcement learning. To make the optimization tractable, we derive an approximate gradient for the upper-level loss. Our framework requires only a sparse set of semantic rigid-body correspondences and eliminates the need for manual tuning by identifying optimal values for a parameterization expressive enough to preserve characteristic motion across different embodiments. Moreover, by integrating retargeting directly with physics simulation, we produce physically plausible motions that facilitate robust imitation learning. We validate our method in simulation and on hardware, demonstrating challenging motions for morphologies that differ significantly from a human, including retargeting onto a quadruped.
Supporting a wide variety of motion styles is critical for creating diverse virtual characters, but current methods either require large stylized datasets or pre-trained models that cannot generalize beyond their training distribution. We present STyMo, a few-shot approach that learns motion style from only seconds of paired data and trains in one to two minutes. Our key insight is to decompose style into two components: a static component capturing time-invariant posture, and a temporal component capturing frame-wise dynamics. This decomposition yields an interpretable system where posture intensity, temporal exaggeration, and per-body-region style can be adjusted at runtime. Furthermore, the reduction in required training data and computation time structurally permits an iterative authoring workflow. To ensure robustness on arbitrary inputs, we further introduce a stylizability gate that automatically prevents artifacts on out-of-distribution motions. We demonstrate results across diverse motion styles, from subtle emotional variations to exaggerated character archetypes, and release our processed paired dataset to facilitate future research. The source code used in this paper can be found at: https://github.com/facebookresearch/STyMo
In this paper, we propose an efficient method for hair and fur rendering that approximates multiple scattering as volumetric light transport, achieving near path-traced visual fidelity at a drastically reduced cost. Multiple scattering among hair fibers is crucial for a realistic appearance, especially for light-colored hair, but it is extremely expensive to simulate directly. Prior approximations, such as dual scattering and fur BSSRDF, improve performance but often fail for dense hair/fur, yielding an overly dry or blurred appearance. Unlike previous volumetric hair rendering solutions, we treat dense hair/fur as a highly anisotropic participating medium to capture soft volumetric illumination, while still using explicit fiber geometry for direct lighting to preserve fine details. We accumulate hair fiber coverage in screen space and stochastically sample a single-scattering event to approximate higher-order scattering. Our method reproduces the rich appearance of hair and fur resulting from multiple scattering, while running about 7–10× faster than path tracing, making it suitable for use in production. We also demonstrate that our approach is robust under a variety of lighting conditions.
Hair is a rich medium of visual and cultural expression, yet its digital modeling remains challenging due to the duality of fluidity and structure. Many existing generative approaches rely primarily on continuous diffusion fields, which entangle global topology with local texture and obscure the semantic and structural organization of hairstyles. To address this, we propose HairGPT, a strand-centric framework that treats strands as generative primitives and formulates realistic 3D hairstyle synthesis as a dual-decoupled autoregressive sequence modeling problem. Our method applies spatial decoupling across semantic scalp regions and structural decoupling along a hierarchical strand representation, progressing from global layout to fine-grained style. We further introduce a geometric tokenizer and region-aware semantic annotations to guide strand-level generation, enabling compositional editing, synthesis of rare and complex hairstyles, and adaptation to stylized domains. By aligning generative modeling with the workflow of digital grooming, HairGPT turns hair generation from opaque texture synthesis into a structured and semantically controllable authoring process, supporting robust semantic conditioning and high-fidelity results across realistic and stylized domains.
Freeform surfaces are popular in architectural design for their expressive aesthetics, yet they remain challenging to fabricate. A common strategy is to approximate them with assemblies of planar polygonal panels in a process we term panelization, enabling cost-effective fabrication from sheet materials such as wood, metal, or glass. Our goal is to compute panelizations that are both geometrically faithful and structurally robust. A two-stage pipeline, i.e. geometric panelization followed by structural optimization, can be effective, but joint optimization may yield better outcomes by letting mechanics influence the panelization itself. The challenges are that the panelization task is inherently combinatorial, and structural objectives depend on stresses that have a complex, global dependence on the geometry and are costly to evaluate. Moreover, improving structural performance must be balanced against preserving the input surface's design intent. To address these challenges, we formulate panelization as a continuous surface flow coupled to stress analysis, by modeling candidates as creased elastic shells. We define a global objective by integrating an arbitrary function of stress over the shell and derive efficient gradients with respect to the surface shape to enable optimization. We further introduce flow terms that better preserve the input shape and show that running BFGS in a Sobolev space substantially outperforms standard BFGS and Sobolev gradient descent. We evaluate our method on architectural models, showing that incorporating stress during panelization yields substantial stress reductions and more robust designs compared to optimizing stress after fixing the panelization.
Despite transformative advances in generative motion synthesis, real-time interactive motion control remains dominated by traditional techniques. In this work, we identify two key challenges in bridging research and production: 1) Real-time scalability: Industry applications demand real-time generation of a vast repertoire of motion skills, while generative methods exhibit significant degradation in quality and scalability under real-time computation constraints, and 2) Integration: Industry applications demand fine-grained multi-modal control involving velocity commands, style selection, and precise keyframes, a need largely unmet by existing text- or tag-driven models. Moreover, a systematic motion design interface for generative models remains absent. To overcome these limitations, we introduce MotionBricks: a large-scale, real-time generative framework with a two-fold solution. First, we propose a large-scale modular latent generative backbone tailored for robust real-time motion generation, effectively modeling a dataset of over 350,000 motion clips with a single model. Second, we introduce smart primitives that provide a unified, robust, and intuitive interface for authoring both navigation and object interaction. Notably, MotionBricks applies to new downstream tasks in a zero-shot manner, where no fine-tuning or task-specific tagging is required. Applications can be designed in a plug-and-play manner like assembling bricks without expert animation knowledge, enabling an accessible interface for applications in animation and robotics. Quantitatively, we show that MotionBricks produces state-of-the-art motion quality on open-source and proprietary datasets of various scales, while also achieving a real-time throughput of 15,000 FPS with 2ms latency. We demonstrate the flexibility and robustness of MotionBricks in a complete production-level animation demo, covering navigation and object-scene interaction across various styles with a unified model. To showcase our framework's application beyond animation, we deploy MotionBricks on the Unitree G1 humanoid robot to demonstrate its flexibility and generalization for real-time robotic control.
Two-phase flows are ubiquitous in nature, exhibiting complex fluid-fluid interactions that challenges numerical simulators. To accurately and efficiently solve two-phase flows, grid-based methods have been widely adopted. Navier-Stokes (NS) solvers consume a small memory footprint, but simultaneously achieving both high performance and low numerical dissipation remains a significant challenge. In contrast, lattice Boltzmann solvers are efficient and have low numerical dissipation, yet they remain memoryintensive, even with state-of-the-art moment-encoding schemes. To date, the simultaneous attainment of high accuracy, exceptional efficiency, and a low memory footprint remains a major challenge in the field. In this paper, we propose a novel two-phase flow solver that achieves this objective. Our work is motivated by extending gas-kinetic scheme (GKS), which is adapted to handle nearly incompressible flows. To allow stable and accurate two-phase flow simulations, we systematically derive a coupled formulation of the GKS method and the phase-field model, incorporating novel mathematical constructs. Combined with robust boundary treatments and specialized techniques for handling turbulent flows, this results a unified framework capable of efficiently simulating two-phase flows, even those with large density contrasts and high Reynolds numbers. Since our formulation is explicit, it achieves exceptional performance when optimized on GPU, making it the fastest kinetic two-phase flow solver to date. Additionally, as it is derived from GKS, it obviates the need to store distribution functions. Thus, it has a small memory footprint, competitive with, or even lower than, that of many NS solvers. As a result, our solver can efficiently simulate complex two-phase flow dynamics at high resolutions using a single commodity GPU. We validate the accuracy of our solver via several benchmark tests, compare its performance with recent methods in various aspects, and demonstrate its capability to replicate a broad range of two-phase flow phenomena, encompassing both typical and large-scale scenarios.
Simulating the dynamic, multiscale interactions between granular materials and multiphase fluids remains a significant computational challenge in computer graphics, as the visual complexity of such mixtures arises from strongly coupled small-scale structures. We present a novel, physically-based simulation framework for sand-water-air mixtures that couples a Lattice Boltzmann Method (LBM) for weakly-compressible two-phase fluids with a Material Point Method (MPM) for granular sand. Our approach is built upon a unified continuum formulation that expresses the governing equations for both fluid phases (air and water) and the granular medium within a consistent framework. To accurately capture the transition of sand from a dry, friction-dominated state to a soaked, sticky medium, we introduce a water retention model that describes how liquid infiltrates and is retained within the granular structure. Furthermore, we enforce volume conservation of the fluids within the mixture, ensuring numerical stability and physical realism. Our robust coupling mechanism enables the simulation of complex phenomena such as sand mobilization, transport, settling, and erosion across a wide range of density ratios. We demonstrate the efficiency of our method through several challenging scenarios, including the breaching of sand-walled basins, sediment-laden flows, and the erosive collapse of sand structures.
Animatable 3D assets, defined as geometry equipped with an articulated skeleton and skinning weights, are fundamental to interactive graphics, embodied agents, and animation production. While recent 3D generative models can synthesize visually plausible shapes from images, the results are typically static. Obtaining usable rigs via post-hoc auto-rigging is brittle and often produces skeletons that are topologically inconsistent with the generated geometry. We present AniGen, a unified framework that directly generates animate-ready 3D assets conditioned on a single image. Our key insight is to represent shape, skeleton, and skinning as mutually consistent S3 Fields (Shape, Skeleton, Skin) defined over a shared spatial domain. To enable the robust learning of these fields, we introduce two technical innovations: (i) a confidence-decaying skeleton field that explicitly handles the geometric ambiguity of bone prediction at Voronoi boundaries, and (ii) a dual skin feature field that decouples skinning weights from specific joint counts, allowing a fixed-architecture network to predict rigs of arbitrary complexity. Built upon a two-stage flow-matching pipeline, AniGen first synthesizes a sparse structural scaffold and then generates dense geometry and articulation in a structured latent space. Extensive experiments demonstrate that AniGen substantially outperforms state-of-the-art sequential baselines in rig validity and animation quality, generalizing effectively to in-the-wild images across diverse categories including animals, humanoids, and machinery. Homepage: https://yihua7.github.io/AniGen_web/
Despite recent advancements in native latent diffusion models for single-view 3D generation, they still suffer from inadequate geometric details due to the global entanglement of VecSet-based representations and inaccurate structural topology arising from single-view ambiguity. Existing solutions often rely on model scaling over large-scale datasets to improve fidelity, which is computationally prohibitive and inefficient in capturing fine-grained details. In this work, we propose Tempo3D, a resource-efficient paradigm for high-fidelity VecSet-based 3D generation via efficient temporal-aware fine-tuning and multi-view latent aggregation. Our approach consists of two critical components: (1) To facilitate robust detail learning, we establish the TempoDetail dataset and design the Time-Segmented LoRA (TS-LoRA) module. TS-LoRA exploits the inherent "structure-to-details" temporal transition of flow-based models to decouple feature learning, significantly bolstering the capacity to produce high-frequency details without large-scale retraining. (2) To mitigate geometric inaccuracies stemming from single-view ambiguity, we propose a multi-view latent aggregation strategy incorporating Soft Flow Trajectory Projection (Soft FTP) and Score-based Velocity Field Aggregation (SVFA). By aggregating reference views derived from Multimodal Large Language Models (MLLMs), these components resolve the intrinsic conflicts between velocity vectors caused by cross-view latent discrepancies, thereby preventing geometric collapse and ensuring directional consistency. Moreover, the integration of MLLMs empowers Tempo3D to support precise semantic 3D editing via natural language prompts while maintaining structural integrity. Experiments demonstrate that Tempo3D outperforms state-of-the-art baselines in generation quality, geometric precision, and editability under a highly efficient computational budget.
This paper presents a highly efficient and robust differentiable flu::id framework, centered on a novel surrogate gradient method that utilizes the flow map structural advantages. Our key insight reveals a significant misalignment between computational intensity and gradient importance during the backward pass. Specifically, we identify a physical duality within the adjoint process, revealing that the cross-step connections inherent in the flow map act as dominant gradient "highways" that propagate sensitivities over long horizons with high fidelity. Leveraging these insights, we develop a surrogate gradient model that retains these critical connections while pruning redundant adjoint computations in a physics-informed manner. Integrated with tailored acceleration techniques, our framework is successfully applied to diverse, challenging optimization tasks characterized by long time horizons and rich vorticity. Results demonstrate significant speedups and memory reductions while maintaining nearly-identical gradients compared to the full-gradient baseline.
The intricate motion arising from fluid-boundary interactions is visually compelling, yet notoriously difficult and computationally expensive to simulate in the presence of complex boundaries. Accurately resolving boundary geometry requires body-fitted grids constructed via cut-cell methods, which often leads to poorly conditioned linear systems in the pressure projection stage and, consequently, prohibitive computational cost. We present FastVEM, an efficient boundary-conforming fluid simulation framework that enables high-fidelity flow-boundary interaction at substantially reduced cost. Computational efficiency is achieved through a coordinated, top-down design spanning numerical discretization, grid construction, and linear solvers. FastVEM adopts a Virtual Element Method (VEM) discretization to robustly enforce incompressibility and boundary conditions on irregular body-fitted grids, and employs a VEM polynomial-space Particle-in-Cell scheme for advection. Complementing this discretization, a convexity-preserving cut-cell strategy is introduced to construct simulation-friendly body-fitted grids. To accelerate pressure projection, we develop a Galerkin geometric multi-grid solver featuring a diffusion-free prolongation operator that prevents coarse-level matrix densification, along with a nested, boundary-aware grid hierarchy that ensures well-posed placement of coarse-level degrees of freedom. Compared to prior cut-cell-based fluid simulators, FastVEM speeds up the computationally dominant pressure projection stage by up to 100×, while robustly handling even more challenging boundary geometries.
Following the success of deep neural networks in 2D pose estimation, reconstruction-based approaches have significantly advanced multi-person 3D pose estimation from sparse multi-view images. These methods typically detect 2D poses independently in each view and then associate them for 3D reconstruction. However, despite strong progress, recent state-of-the-art methods still face critical limitations: 1) They often depend on global optimization over a large and complex set of multi-view 2D joints to jointly infer 3D poses for all individuals, making the process highly complex and prone to suboptimal solutions; 2) Their tight coupling with the bottom-up detector OpenPose hinders the use of more advanced top-down or single-stage 2D pose estimators and restricts the integration of richer instance-level cues learned by these models. To address these limitations, we propose TwinPose, a novel framework that alleviates the complexity of global pose inference by optimizing within person-specific 3D pose subspaces, while fully supporting diverse 2D pose detectors and effectively leveraging pose-instance cues. The key idea is to introduce a twin pose — a 3D counterpart of each 2D pose — that inherits its instance representation and aggregates geometrically consistent 2D joints from other views. All twin poses are unified in a common 3D space, where those belonging to the same individual naturally share a number of bones. This structural property enables association by counting shared bones, forming person-specific subspaces from which each individual's 3D pose can be inferred independently in an efficient and robust manner. Extensive experiments demonstrate that TwinPose achieves state-of-the-art performance in both accuracy and efficiency across multiple public and proprietary datasets. Importantly, it is fully detector-agnostic, allowing seamless integration with current and future advances in 2D pose estimation while remaining highly robust to noisy or imperfect 2D predictions. Project page with code and additional resources: https://github.com/zgspose/TwinPose
Recent work in 3D deep learning has demonstrated that unsigned distance functions (UDFs) are a useful representation for 3D reconstruction and shape generation because they can represent surfaces with arbitrary topology. However, extracting meshes that preserve the intended topology, especially in the presence of non-manifold structures, remains challenging. We present DCx, an extension of the standard Dual Contouring (DC) method which was originally proposed for isosurface extraction from signed distance functions (SDFs). Standard DC operates on individual voxels and inserts one vertex per active cube, where activation is determined by detecting sign changes. To address the lack of sign information in UDFs, DCx adopts an optimization-based strategy for determining active cubes. It operates on each 2 × 2 × 2 voxel block, referred to as an expanded cube, and introduces a voxel-to-mesh lookup table that stores connectivity patterns based on local voxel configurations. This enables efficient triangle extraction using predefined templates. These changes improve upon DC by avoiding failure cases caused by unreliable active-cube detection in UDFs and by correcting mesh connections in non-manifold regions. As a result, DCx supports the extraction of both manifold and non-manifold surfaces from neural UDFs. DCx is conceptually simple and easy to implement. Experimental results show that DCx produces meshes with higher accuracy in a more robust way than existing methods, particularly on shapes with complex geometry or non-manifold structures. The source code is available at http://github.com/jjjkkyz/DCx.