We introduce Soft Anisotropic Diagrams (SAD), an explicit and differentiable image representation parameterized by a set of adaptive sites in the image plane. In SAD, each site specifies an anisotropic metric and an additively weighted distance score, and we compute pixel colors as a softmax blend over a small per-pixel top-K subset of sites. We induce a soft anisotropic additively weighted Voronoi partition (i.e., an Apollonius diagram) with learnable persite temperatures, preserving informative gradients while allowing clear, content-aligned boundaries and explicit ownership. Such a formulation enables efficient rendering by maintaining a per-query top-K map that approximates nearest neighbors under the same shading score, allowing GPU-friendly, fixed-size local computation. We update this list using our top-K propagation scheme inspired by jump flooding, augmented with stochastic injection to provide probabilistic global coverage. Training follows a GPU-first pipeline with gradient-weighted initialization, Adam optimization, and adaptive budget control through densification and pruning. Across standard benchmarks, SAD consistently outperforms Image-GS and Instant-NGP at matched bitrate. On Kodak, SAD reaches 46.0 dB PSNR with 2.2 s encoding time (vs. 28s for Image-GS), and delivers 4–19× end-to-end training speedups over state-of-the-art baselines. We demonstrate the effectiveness of SAD by showcasing the seamless integration with differentiable pipelines for forward and inverse problems, efficiency of fast random access, and compact storage. You can find the code here: https://luckyiyi.github.io/SAD.
论文检索
输入标题、作者或关键词，从 2,132 篇学术成果中精准定位
Constant mean curvature surfaces are æsthetically appealing geometric objects with applications in physics, differential geometry, and architecture. We present a straightforward discretization of constant mean curvature surfaces based on the classical observation that their Gauß maps are harmonic. Our construction is elementary—requiring only discrete Dirichlet energy minimization and a Poisson solve—yet it exactly mirrors this aspect of the smooth theory. A discrete analog of conjugation produces discrete constant Gauß curvature surfaces. Their unit offsets are discrete CMC surfaces with a conformal parameterization. Additionally, we introduce a novel Möbius-invariant discretization of the Dirichlet energy for sphere-valued maps on dual meshes that is derived from the discrete Willmore energy. It is more robust than standard formulations based on inverse cotangent weights. Our approach to the construction of constant mean curvature surfaces provides direct control over tangent planes along a boundary, if present, and naturally handles closed and periodic examples. We demonstrate the approach on a range of free-boundary, symmetric, and periodic CMC surfaces.
To enable differentiation with respect to object geometries, boundary path integrals—central to physics-based differentiable rendering—must be estimated numerically. Although their mathematical formulation is well established, designing efficient and robust numerical estimators remains challenging. Most state-of-the-art boundary sampling methods rely on primary-sample-space guiding, which tends to break down on finely tessellated geometries; reparameterization-based alternatives, meanwhile, often incur high variance and/or significant computational overhead. In this paper, we introduce a simple, robust, and consistent solution to this problem. At the core of our approach is a novel formulation of the boundary integral based on kernel-density estimation. Much like photon mapping, we slightly expand the measure-zero domain of integration with a kernel, sidestepping the need to sample directly on a delta-function region in path space. To our knowledge, this is the first such application of kernel-density methods for boundary integral evaluation. We validate our method by comparing its derivative estimates against finite differences (FD), and further demonstrate its practical utility by benchmarking against several state-of-the-art baselines in synthetic inverse-rendering scenarios.
Differentiable path tracing offers a principled route to recovering physical material and lighting parameters, but the combination of high variance and poor numerical conditioning often makes it too brittle to use in practice. This is especially the case when lighting is altogether unknown, or when the scene contains complex light transport effects. Prior work recently showed that the variance reduction provided by a radiance cache can alleviate these challenges. We revisit the combination of inverse rendering and radiance caching with a twist, by introducing a spatial blending field that locally interpolates between the cache and standard unbiased estimators. Recursive application of this idea yields a rich design space of evaluation strategies and inter-estimator consistency losses; we map this space and identify effective components. A surprising property of the resulting algorithm is that it can accurately recover material parameters even when the lighting is not uniquely identifiable from the observations. Our experiments demonstrate significant improvements in speed and robustness over prior work, making a strong case for including radiance caching as a standard component of future physically based inverse rendering systems.
Discrete X-ray tomography reconstructs the internal structure of an object from X-ray projections, assuming that the volume is composed of a discrete set of known materials (e.g., steel, aluminum, and air). This is generally straightforward when many projections are available but becomes increasingly ill-posed as their number decreases. Discrete tomography has been extensively studied over the past five decades, resulting in a range of mature reconstruction algorithms. In this work, we introduce a new reconstruction method that draws inspiration from both classical computed tomography and recent advances in inverse rendering, demonstrating that a remarkably simple gradient-based inversion can significantly surpass the reconstruction quality of standard methods such as SIRT, DART, and TVR-DART. Our method represents each 3D location as a probability distribution over the set of known materials and minimizes a volumetric loss that encourages consistency with the measured projections. It supports nonlinear effects such as volumetric scattering and is simple to optimize and parallelize on compute accelerators. We evaluate our method on challenging 2D and 3D benchmarks, demonstrating superior performance particularly in sparse and limited-angle scenarios, where traditional techniques struggle with ambiguity.
Mixed Reality (MR) headsets promise a future of immersive telepresence where virtual humans blend indistinguishably into their real or virtual surroundings. Achieving this vision requires a method capable of capturing a user's motion, estimating their appearance under novel lighting, and understanding the surrounding environment - all from the constrained viewpoint of a head-mounted display (HMD). Existing approaches treat these as isolated problems: they either focus on driving avatars with baked-in lighting or rely on complex studio setups for relighting. In this paper, we present EgoRelight, a holistic framework for egocentric telepresence that simultaneously captures full-body human performance, synthesizes photorealistic and relightable appearance, and estimates high dynamic range (HDR) environment maps from a single HMD. First, to ensure accurate motion and surface reconstruction, we propose an egocentric perception module that leverages stereo down-facing cameras to extract dense depth maps, which serve as robust geometric control signals to drive a mesh-based avatar. Second, we introduce a novel neural appearance model that learns to synthesize view-dependent specular and view-independent diffuse shading separately. By employing a specialized ray-sampling strategy, our model generalizes to unseen illumination without relying on restrictive analytical BRDF priors. Third, we enable seamless avatar integration into the physical world via a test-time inverse rendering process, which recovers an HDR environment map by matching the pre-trained avatar's appearance to the live egocentric camera observations. We demonstrate the efficiency of our system through a social telepresence application, where remote users are coherently relit according to their local physical environment. Extensive experiments show that our individual components and the integrated system significantly outperform the best combinations of state-of-the-art baselines in both geometric accuracy and rendering as well as relighting fidelity. Further details can be found on our project page1.
Lattice metamaterials enable lightweight, multifunctional structures, yet homogenization-based evaluation of their effective properties remains computationally expensive. Neural surrogates offer speed but often lack the accuracy and stability required for engineering-grade simulations. We introduce GMT, a Geometric Multigrid Transformer - a neural solver with high numerical fidelity for fast and reliable lattice homogenization. GMT achieves architectural alignment with Geometric Multigrid (GMG) by restructuring Point Transformer V3 to operate across sparse GMG hierarchies, capturing long-range dependencies and cross-level interactions essential for multigrid convergence. To enforce physical consistency, GMT incorporates physics-aware positional encoding for strict enforcement of periodicity and predicts both the finest-level solution and multi-level residual corrections. These predictions deliver a spectrally-aligned initialization, enabling end-to-end training under physics-informed and solver-aware losses and requiring only a single GMG V-cycle refinement to reach convergence. This fusion of neural prediction and numerical rigor achieves relative residual errors of 10-5 with a 160× speedup over state-of-the-art GPU-based solvers at equivalent accuracy - particularly at high resolutions (e.g., 5123), where traditional methods become most costly. We validate GMT across mechanical and thermal domains, demonstrate robust generalization to unseen geometries and non-periodic settings, and showcase scalability to high resolutions - enabling real-time design iteration, multi-scale simulations, high-throughput material discovery, and inverse design.
We propose a system for differentiating through solutions to geometry processing problems. Our system differentiates a broad class of geometric algorithms, exploiting existing fast problem-specific schemes common to geometry processing, including local-global and ADMM solvers. It is compatible with machine learning frameworks, opening doors to new classes of inverse geometry processing applications. We marry the scatter-gather approach to mesh processing with tensor-based workflows and rely on the adjoint method applied to user-specified imperative code to generate an efficient backward pass behind the scenes. We demonstrate our approach by differentiating through mean curvature flow, spectral conformal parameterization, geodesic distance computation, and as-rigid-as-possible deformation, examining usability and performance on these applications. Our system allows practitioners to differentiate through existing geometry processing algorithms without needing to reformulate them, resulting in low implementation effort, fast runtimes, and lower memory requirements than differentiable optimization tools not tailored to geometry processing.
Stochastic Monte Carlo solvers for partial differential equations (PDEs) recently gained popularity in computer graphics, finding applications in geometry processing, rendering, simulation, and visualization. At present, there exists no Monte Carlo solver for the rendering of biharmonic diffusion curves, an artist-friendly smooth vector graphics primitive. The fourth-order biharmonic equation of biharmonic diffusion curves can be split into two second-order PDEs, namely a Laplace and a Poisson equation. However, since biharmonic diffusion curves set Dirichlet and inhomogeneous Neumann conditions at the same time, these two second-order PDEs are tightly coupled and can hence not be solved directly. We propose to treat the rendering of biharmonic diffusion curves as an inverse problem, in which the Dirichlet data of the Laplace equation is unknown. We formulate a variational energy optimization, such that the user-defined boundary conditions are met. Thereby, the necessary gradients are estimated stochastically by solving two second-order problems with Dirichlet boundary conditions only.
A common process in authoring digital scenes for games, films, and virtual environments is for artists to construct 3D geometry that matches a 2D perspective reference image. In the case of the bare-earth terrain, this is typically a manual process since, unlike for trees and buildings, few inverse reconstruction methods currently exist. To address this, we introduce a method for automatically inferring a detailed, consistent, and complete terrain heightmap from a single photographic image. Our initial phase involves extracting camera parameters and a 3D pointmap from the input image, which is then transformed into a heightmap. However, this only recovers the unoccluded portions of the terrain visible from the perspective of the image. The next phase thus entails the generation of plausible occluded regions using a diffusion model trained on terrain elevation data. The entire process is guided by three consistency principles: geomorphological consistency (the features of the occluded terrain resemble the visible portions), hydrological consistency (the river network is uninterrupted and flows reliably), and view consistency (the shape of the rendered terrain accurately matches the input image). We demonstrate that our method obeys these principles, reliably generates terrains across various scales, and integrates with scene authoring workflows.
Neutron transport simulation plays a central role in nuclear engineering, yet its inverse problems—such as shielding design and sensitivity analysis—remain extremely challenging. The governing transport equation shares a common mathematical foundation with the volume rendering equation used for participating media in computer graphics. Recent advances in differentiable rendering have demonstrated that inverse problems—such as recovering scene properties from observations—can be efficiently solved through gradient-based optimization, where the gradients are estimated through Monte Carlo methods. Inspired by these developments, we extend differentiable rendering techniques to neutron transport problems, enabling inverse design and sensitivity analysis in nuclear systems. To adapt these methods to the neutron transport problem, we generalize the automatic differentiation of Monte Carlo light transport simulation to multi-energy neutron transport; propose a new simplified way to handle moving geometric discontinuities; and propose new estimators for derivatives of volume integrals. We demonstrate the effectiveness of the proposed framework on several model inverse problems, including gradient-based optimization of multi-group neutron moderation, sensitivity analysis of material properties, and diagnostic sensor shape optimization. These results show that differentiable Monte Carlo neutron transport can support optimization and analysis tasks, opening new opportunities for inverse design in nuclear shielding and related applications.
We propose a system to optimize parametric designs subject to radiation pressure, i.e., the effect of light on the motion of objects. This is most relevant in the design of spacecraft, where radiation pressure presents the dominant non-conservative forcing mechanism, which is the case beyond approximately 800 km altitude. Despite its importance, the high computational cost of high-fidelity radiation pressure modeling has limited its use in large-scale spacecraft design, optimization, and space situational awareness applications. We enable this by offering three innovations in the simulation, in representation and in optimization: First, a practical computer graphics-inspired Monte-Carlo (MC) simulation of radiation pressure. The simulation is highly parallel, uses importance sampling and next-event estimation to reduce variance and allows simulating an entire family of designs instead of a single spacecraft as in previous work. Second, we introduce neural networks as a representation of forces from design parameters. This neural proxy model, learned from simulations, is inherently differentiable and can query forces orders of magnitude faster than a full MC simulation. Third, and finally, we demonstrate optimizing inverse radiation pressure designs, such as finding geometry, material or operation parameters that minimizes travel time, maximizes proximity given a desired end-point, minimize thruster fuel, trains mission control policies or allocated compute budget in extraterrestrial compute.
Capturing the full plenoptic light transport across spatial, angular, and temporal dimensions has long been a pursuit in computational imaging, yet it remains fundamentally constrained by the high dimensionality of the sampling space and the physical inaccessibility of scene regions due to self-occlusions. While time-resolved imaging records the temporal axis, existing methods are bottlenecked by the combinatorial complexity of the plenoptic function. This high dimensionality makes dense omni-dimensional sampling physically prohibitive. Simultaneously, tight coupling between illumination and viewpoint in current systems also precludes the full acquisition of plenoptic light transport. In this work, we present GenPIE, a Generative Plenoptic Imager designed to bridge the gap between sparse physical observations and high-dimensional light transport. We introduce a decoupled laser-detector hardware setup that enables independent control over illumination and detection, allowing for active probing of indirect light paths. To overcome the ill-posedness of sparse sampling and physical blind spots, we propose a generative inverse transient rendering framework. Our approach leverages 3D foundation models to provide strong semantic and 3D geometric priors for initialization, which are subsequently refined through a differentiable transient path tracer to ensure physically grounded adherence to the Transient Rendering Equation. We demonstrate that GenPIE supports a range of applications that are challenging for steady-state or purely neural methods, including disentangling multi-bounce light transport directly from captured transient videos, time unwarping, and time-resolved relighting. The project page is at https://wangzh1.github.io/GenPIE.
High-quality facial appearance capture has traditionally required costly studio recording. Recent works consider an in-the-wild smartphone-based setup; however, their model-based inverse rendering paradigm struggles with the complex disentanglement of reflectance from unknown illumination. To bridge this gap, we propose to shift the paradigm into training a powerful delighting network as a prior to constrain the optimization. We leverage the OLAT dataset and the rendered Light Stage scans for training, and propose Dataset Latent Modulation (DLM) to seamlessly integrate these heterogeneous data sources. Specifically, by conditioning the core network on learnable source-aware tokens, we decouple dataset-specific styles from physical delighting principles, enabling the emergence of a delighting prior that outperforms existing proprietary models. This powerful delighting prior enables a simple and automatic appearance capture pipeline that achieves high-quality reflectance estimation from casual video inputs, outperforming prior arts by a large margin. Furthermore, we leverage our appearance capture method to transform the multi-view NeRSemble dataset into NeRSemble-Scan, a large-scale collection of 4K-resolution relightable scans. By open-sourcing our model and the NeRSemble-Scan dataset, we democratize high-end facial capture and provide a new foundation for the research community to build photorealistic digital humans.
Volumetric additive manufacturing promises near-instantaneous fabrication of 3D objects, yet achieving high fidelity at the micro-scale remains challenging due to the complex interplay between optical diffraction and chemical effects. We present Single-View Holographic Volumetric Additive Manufacturing (SHVAM), a mechanically static system that shapes volumetric dose distributions using time-multiplexed, phase-only holograms projected from a single optical axis. To achieve high resolution with SHVAM, we formulate hologram synthesis as a coupled inverse problem, integrating a differentiable wave-optical forward model with a simplified photochemical model that explicitly captures inhibitor diffusion and non-linear dose response. Optimizing hologram sequences under these coupled constraints allows us to pre-compensate for chemical blur, yielding higher print fidelity than optical-only optimization. We demonstrate the efficacy of SHVAM by fabricating simple 2D and 3D structures with lateral feature sizes of approximately 10 μm within a 0.8 mm × 0.8 mm × 3 mm volume in seconds.
Representing complex geometry as a displacement function defined over a coarse base mesh enables compact storage and accelerated rendering. The core challenge in converting detailed triangle meshes into this representation is computing base meshes that have as few triangles as possible, while also supporting displacement functions that accurately approximate the input. Accurate approximation requires the supported displacement functions to bijectively map the input surface onto the base with low parametric distortion. We observe that this distortion can be measured by evaluating the pointwise Jacobian of the displacement functions. Our new DJM (Displacement Jacobian Metric)-based base-mesh construction method uses the Jacobian of the displacement functions to guide base mesh computation, enabling us to outperform prior approaches in terms of accuracy to size trade-off. We achieve this goal by proposing a variant of the QEM-based simplification scheme that constrains the displacement mapping between the input and the base to be bijective and low distortion (defined as satisfying a lower bound on the mapping Jacobian). When evaluating and encoding the displacement maps, we avoid unreliable ray-mesh intersections by explicitly storing the mapping between the input mesh and the base throughout the construction process, and use this mapping within a robust inverse barycentric displacement solver to obtain dense base-to-mesh correspondences to assist all computations. We demonstrate DJM to outperform alternative schemes in terms of reconstruction accuracy to size trade-off, and demonstrate its robustness and usability for micromesh based rendering and neural encoding.
Recent radiance-field-based reconstruction methods, such as NeRF and 3DGS, achieve high visual fidelity for indoor scenes, but often break down under scene editing due to baked illumination and the lack of explicit light transport. In contrast, inverse path tracing methods based on mesh representations enforce correct light transport but require highly accurate geometry, making them difficult to apply robustly to real indoor scenes. We present Emission-Aware Gaussians and Path Tracing (EAG-PT), a method for physically based reconstruction and rendering of indoor scenes using a unified 2D Gaussian representation, targeting editable diffuse global illumination. Our approach consists of three key ideas: (1) representing indoor scenes with 2D Gaussians as a transport-friendly geometric proxy that avoids explicit mesh reconstruction; (2) explicitly separating emissive and non-emissive components during reconstruction to support editing; and (3) decoupling reconstruction from final rendering by using efficient single-bounce optimization and high-quality multi-bounce path tracing, respectively. Experiments on synthetic and real indoor scenes show that EAG-PT produces more natural and physically consistent edited renderings than radiance-field reconstructions, while preserving finer geometric detail and avoiding mesh-induced artifacts compared with mesh-based inverse path tracing. These results highlight the potential of our approach for applications such as interior design, XR content creation, and embodied AI.
Computing injective, flip-free embeddings of simplicial meshes is a fundamental challenge in geometry processing. While the Total Lifted Content (TLC) energy provides a smooth functional with theoretically guaranteed injective global minima, its non-convex nature presents significant optimization hurdles. In this work, we propose a Nonlinear Multigrid (MG/Opt) framework specifically tailored for efficient TLC energy minimization. Our approach leverages a topology-preserving mesh hierarchy to rapidly eliminate low-frequency errors on coarse scales while maintaining rigorous gradient consistency between hierarchical levels through linear correction terms. To address the persistent challenge of local inversions, we introduce a targeted smoothing strategy and an adaptive parameter scheduling mechanism that balances global convergence speed with local injectivity precision. Experimental evaluations across extensive 2D and 3D benchmark datasets demonstrate the robustness of our approach. Our method achieves a 100% convergence success rate across all tested 2D cases and all but one 3D dataset (the VOLMAP benchmark). Furthermore, our MG/Opt framework delivers substantial performance gains, achieving up to 63 × speedup on high-resolution subdivided datasets compared to traditional solvers. These results confirm that our multigrid strategy provides a scalable and reliable solution for large-scale injective mapping problems. Code and data for this paper are at https://github.com/distmeshM/MGOpt_TLC