Discrete X-ray tomography reconstructs the internal structure of an object from X-ray projections, assuming that the volume is composed of a discrete set of known materials (e.g., steel, aluminum, and air). This is generally straightforward when many projections are available but becomes increasingly ill-posed as their number decreases. Discrete tomography has been extensively studied over the past five decades, resulting in a range of mature reconstruction algorithms. In this work, we introduce a new reconstruction method that draws inspiration from both classical computed tomography and recent advances in inverse rendering, demonstrating that a remarkably simple gradient-based inversion can significantly surpass the reconstruction quality of standard methods such as SIRT, DART, and TVR-DART. Our method represents each 3D location as a probability distribution over the set of known materials and minimizes a volumetric loss that encourages consistency with the measured projections. It supports nonlinear effects such as volumetric scattering and is simple to optimize and parallelize on compute accelerators. We evaluate our method on challenging 2D and 3D benchmarks, demonstrating superior performance particularly in sparse and limited-angle scenarios, where traditional techniques struggle with ambiguity.
论文检索
输入标题、作者或关键词，从 1,093 篇学术成果中精准定位
A common process in authoring digital scenes for games, films, and virtual environments is for artists to construct 3D geometry that matches a 2D perspective reference image. In the case of the bare-earth terrain, this is typically a manual process since, unlike for trees and buildings, few inverse reconstruction methods currently exist. To address this, we introduce a method for automatically inferring a detailed, consistent, and complete terrain heightmap from a single photographic image. Our initial phase involves extracting camera parameters and a 3D pointmap from the input image, which is then transformed into a heightmap. However, this only recovers the unoccluded portions of the terrain visible from the perspective of the image. The next phase thus entails the generation of plausible occluded regions using a diffusion model trained on terrain elevation data. The entire process is guided by three consistency principles: geomorphological consistency (the features of the occluded terrain resemble the visible portions), hydrological consistency (the river network is uninterrupted and flows reliably), and view consistency (the shape of the rendered terrain accurately matches the input image). We demonstrate that our method obeys these principles, reliably generates terrains across various scales, and integrates with scene authoring workflows.
Holography offers unique advantages for delivering perceptual realism while preserving compact form factors in VR/AR. Its perceptual quality, however, hinges on encoding rich wavefronts of photorealistic scenes into interference patterns and then incoherently multiplexing the resulting wave fields for perception. Existing CGH paradigms decouple radiance estimation from wave propagation by pre-rendering radiance on discretized scene sectors. This separation between radiometric and wave-optical computation inherently limits the range of focus cues and visual effects that can be faithfully reproduced, including depth- and view-continuity, and physically based material behaviors such as glossy or mirror-like reflection and refraction. We present a physically accurate yet computationally efficient wave optics rendering framework leveraging path tracing to encode full 3D visual cues into phase holograms. Specifically, we employ a Monte Carlo method to solve both the rendering equation and the Rayleigh-Sommerfeld integral simultaneously. Our algorithm is fully compatible with modern graphics techniques and can generate multiple time-multiplexed random holograms with minimal additional time cost via Path Reuse. By employing a fast approximation with an ambient radiance cache, we realize an order of magnitude convergence speed improvement. The resulting coherent wave fields that inherently encode comprehensive visual effects are converted into phase-only holograms under complex-amplitude supervision. Through extensive simulations and experimental validations on a spatial light modulator-based display prototype, we demonstrate faithful holographic reconstructions of natural 3D cues and complex materials, including realistic defocus blur, view-dependent effects, as well as appearance highlights and reflections.
Online 3D reconstruction from monocular image sequences is a challenging and ongoing research topic. 3D Gaussian Splatting (3DGS), leveraging its high-quality real-time rendering capability, empowers online 3D reconstruction to represent dense scenes with enhanced expressiveness, and thus holds great promise for a wide range of applications such as robotics and AR/VR. However, existing online 3DGS methods still suffer from some key challenges: fragile camera pose estimation due to the lack of global optimization, and low optimization efficiency in large-scale or long-sequence scenarios. To address these issues, we propose a robust and efficient online voxelized 3DGS reconstruction framework integrated with global Sim(3) optimization, which enables reliable camera tracking and efficient global loop closure for both camera poses and voxelized 3DGS. To accelerate the convergence of the voxelized 3DGS, we further introduce a color residual learning strategy, which not only boosts optimization speed but also enhances rendering quality. Extensive experiments on diverse indoor and outdoor datasets demonstrate that our method achieves state-of-the-art performance in both camera pose estimation accuracy and rendering quality, while retaining real-time efficiency. Additionally, we develop and deploy a real-world UAV-based active reconstruction system grounded on our proposed method, validating its robustness and generalizability for practical online 3D reconstruction tasks. Our code and data are available at https://github.com/TrickyGo/MoonSplat..
3D Gaussian Splatting (3DGS) has become the method of choice for reconstructing and real-time rendering of captured scenes. To capture a scene with good visual quality, continuous image sequences are usually combined with out-of-order shots for better scene coverage. Structure from motion can reconstruct such captures, but only after they are all available and often with high computational cost. Incremental reconstruction methods – often derived from SLAM solutions – provide immediate feedback, but cannot handle the out-of-order capture we require. We provide the first immediate feedback solution for such radiance field capture that provides global consistency. We first introduce a method for fast matching in out-of-order sequences, by repurposing visual place recognition models and a covisibility graph, and provide an efficient way to find highly connected keyframes, improving quality even for ordered sequences. We show how these steps – together with GPU optimization and careful Gaussian primitive placement – provide fast local reconstruction, in our challenging radiance field reconstruction case. We then introduce a novel cluster-based method, again using the covisibility graph, to provide efficient loop closure that does not require sequential input. Finally, to handle large scenes in our context, we introduce a progressive hierarchy that allows our method to scale to large environments, without compromising efficiency. Our results show we provide immediate feedback 3DGS reconstruction with good visual quality in several datasets, with up to thousands of input images.
Radar neural reconstruction methods have recently achieved robust 3D scene occupancy from radar measurements alone, as they provide metric depth and are insensitive to adverse weather and low light. However, while these methods can recover some 3D geometry, their input radar data mixes information across elevation into a 2D range-azimuth measurement. This fundamentally limits their elevation resolution, especially in automotive scenes with limited vertical baselines. Camera images offer the opposite tradeoff: they contain strong, high resolution cues for object elevation but struggle with accurate depth and in adverse conditions. We propose CaRaFe, a method that leverages the complementary strengths of camera and radar for 3D reconstruction in challenging urban settings. CaRaFe employs a single multi-modal neural field, relying on conventional novel view synthesis as a supervision signal. Radar supervision provides a valuable geometric constraint to camera rendering that reduces shape-radiance ambiguity, while camera supervision allows for more accurate object elevation disambiguation and fewer missing structures in regions weakly observed by radar. We validate CaRaFe across diverse in-the-wild driving scenes, demonstrating favorable reconstruction quality over both radar and camera methods. Code for this paper is available at light.princeton.edu/carafe.
Imaging objects hidden outside the direct line of sight expands the effective field of view and is critical for applications such as autonomous driving and robotic perception. Despite impressive progress in time-of-flight (ToF)-based non-line-of-sight (NLOS) imaging, real-world deployment remains challenging because practical measurements are often collected over spatially limited, arbitrarily shaped relay regions—conditions that violate the planar-wall and dense-sampling assumptions made by most existing methods. To address these limitations, we propose a LOS-guided NLOS imaging pipeline that imposes no geometric assumptions on the relay surface and naturally supports both confocal and non-confocal configurations. Our method represents the hidden scene using 3D Gaussian primitives and couples them with an efficient, differentiable transient rendering model, enabling end-to-end optimization directly from measured transients. We validate our approach on real-world measurements from both a public dataset and a custom-built capture system. Across settings, our method achieves state-of-the-art reconstruction fidelity under spatially limited, sparsely sampled conditions, and significantly outperforms existing methods on complex, arbitrary relay surface geometries. We release our code and datasets at nlos-3d-gtr.
World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintaining long-term content consistency when scenes are revisited and enabling precise camera control from user-specified inputs. Existing methods based on explicit 3D reconstruction often compromise flexibility in unbounded scenarios and struggle to preserve fine-grained structures. Alternative methods rely directly on previously generated frames without establishing explicit spatial correspondence, thereby limiting controllability and consistency. To address these limitations, we present UCM, a novel framework for unified modeling of long-term memory and precise camera control via a time-aware positional encoding warping mechanism. To reduce computational overhead, we design an efficient dual-stream diffusion transformer for high-fidelity generation. Moreover, we introduce a scalable data curation strategy that utilizes point-cloud-based rendering to simulate scene revisiting, enabling training on over 500K monocular videos. Extensive experiments on real-world and synthetic benchmarks demonstrate that UCM significantly outperforms state-of-the-art methods on long-term scene consistency, while achieving precise camera controllability in high-fidelity video generation. Our code is released at https://humanaigc.github.io/ucm-webpage/.
Reconstructing dynamic fluids from sparse-view videos is a severely under-constrained problem due to complex volumetric visibility and turbulent, topology-changing motion. Current dynamic deformation-based 3D Gaussian Splatting (3DGS) methods use heuristic image-space warps, which often overfit limited observations via non-physical primitive scaling or drifting, producing floaters and geometric distortions. To address these challenges, we propose a hybrid, physics-aware fluid representation that injects physical constraints directly into the 3DGS pipeline. We develop a Beer–Lambert volumetric model mapping Gaussian primitives to a continuous, physically-grounded density field, bridging optical observations with fundamental physical laws. A physically-guided optimization evolves primitives under a global, volume-preserving velocity field, avoiding arbitrary per-particle deformations and effectively preserving volume while recovering fine-scale turbulent details. This unified formulation tightly couples fluid dynamics with differentiable rendering, enabling high-quality, joint predictions of velocity and density. Experiments on synthetic and real smoke datasets demonstrate that our approach surpasses state-of-the-art methods in both visual fidelity and physical consistency, achieving stable 3D reconstructions even from sparse views.
This paper presents a modern, comprehensive system for real-time light field path tracing, addressing the prohibitive computational overhead inherent in driving high-density 3D displays. While offering immersive glasses-free 3D experiences, these displays necessitate the synthesis of massive angular information across numerous viewpoints to satisfy their optical requirements. Conventional rendering pipelines often overlook the hardware’s intrinsic spatial-angular mapping, resulting in redundant computations and extreme ray counts that preclude interactive performance. We resolve this by reformulating light field synthesis as a sparse signal reconstruction task on a high-dimensional manifold. We introduce a unified algebraic framework governed by three mathematical primitives that respectively decouple rendering logic from heterogeneous optical hardware, establish deterministic geometric connectivity across disparate viewpoints, and enable atomic access to unstructured sparse data. Leveraging this formalism, we propose a geometry-driven active sensing pipeline. By modulating a stochastic checkerboard sampling pattern with local disparity cues and coupling this with 5D temporal accumulation and unified spatio-angular integration, our method proactively allocates ray budgets to geometrically complex regions. We demonstrate a complete end-to-end system capable of driving 8K light field displays at interactive frame rates using a single consumer GPU. Our approach not only significantly outperforms existing baselines in computational efficiency and reconstruction fidelity but also naturally supports complex modern light transport effects. Project Page: https://coronaengine.github.io/ssat-page
Reconstructing high-fidelity, animatable, and relightable 3D hand avatars from a single RGB image is a challenging yet critical task for immersive VR/AR applications. State-of-the-art hand reconstruction methods achieve impressive reconstruction and relighting results, but they mostly leverage dense observations, such as multi-view images or monocular video sequences and rely on per-scene optimization. Moreover, it is difficult for these methods to generate plausible appearance in occluded regions. In contrast, existing single-view hand reconstruction methods typically struggle to disentangle global illumination, resulting in textures with baked-in shading artifacts. To address these challenges, we propose FLASHand, the first feed-forward relightable and animatable 3D hand reconstruction model from a single RGB image. Our method bridges the gap between efficiency and fidelity, enabling instant creation of personalized hand avatars with disentangled appearance that can be rendered under novel lighting and poses. To ensure plausible geometry generation and achieve high-fidelity animation and relighting, we introduce a mesh-based disentangled 2D Gaussian splatting hand representation. We leverage the NIMBLE parametric prior to define a canonical attribute space, and define geometry and appearance attributes explicitly. We then propose the Feed-forward Hand Attributes Generator (FHAG) to predict these attributes from a single image. By leveraging a cross attention module to fuse canonical geometric information with input image features, FHAG effectively lifts pixel-level visual cues into the 3D hand canonical space, directly generating spatially-aligned geometry and appearance attributes. This allows us to bypass iterative optimization and instantly reconstruct a high-fidelity hand avatar from a single RGB image. Extensive experiments on both synthetic and in-the-wild datasets demonstrate that FLASHand achieves state-of-the-art performance in novel view synthesis and supports real-time animation and relighting. Code and data are available at https://github.com/IGLICT/FLASHand.
Trying different fabrics on existing garments is a widely applicable problem in digital fashion and computer graphics. A comprehensive transformation involves both material reflectance and geometric deformation from fabric drape. In this work, we focus on the visual aspects of this challenge and simplify fabric try-on to a re-texturing task that replaces garment materials while preserving the original geometry and illumination. Prior approaches perform garment re-texturing via 3D or UV-space reconstruction and rendering, making them sensitive to reconstruction accuracy and rendering fidelity. Recent diffusion-based material transfer methods either lack fine-grained geometric and material control or suffer from domain gaps due to training on synthetic rendered data. We propose a fabric try-on framework that leverages the generative priors of modern image editing models. Motivated by the in-context generation capability of Multimodal Diffusion Transformers, we reformulate garment re-texturing as a two-stage process consisting of fabric removal and fabric application via an intermediate material-normalized image. We further introduce a real-image data curation pipeline and a context-aware tile augmentation strategy, enabling coherent and photorealistic fabric try-on from a single image. Extensive experiments show that our method achieves high-quality, controllable fabric transfer while preserving garment geometry and illumination, without requiring costly reconstruction or rendering pipelines. Our project is available at: https://style3d.github.io/fabric_tryon.
Fluorescence molecular tomography (FMT) serves as a pivotal modality for preclinical tumor screening. While single-view FMT offers distinct advantages in data acquisition efficiency and cost-effectiveness, the scarcity of projection views severely exacerbates photon scattering-induced depth ambiguity, rendering 3D volumetric recovery a highly ill-posed inverse problem. To address these challenges, we propose a physics-inspired spectral topology aware reconstruction network (STAR-Net). Specifically, STAR-Net establishes a synergistic framework: initially, a frequency domain decoupling strategy is introduced to simulate the physical characteristics of diffuse light fields; building on this, a differentiable inverse spectral gating (DISG) mechanism is utilized to explicitly impose low-pass spectral regularization for precise depth recovery; and further, a dual-domain synergistic module is integrated to dynamically fuse spatial and frequency features, achieving high-fidelity detail preservation. Extensive experiments on the Digimouse benchmark demonstrate that the proposed STAR-Net achieves the highest dice coefficient under single-view conditions, validating that explicit spectral topology modeling is a powerful paradigm for mitigating depth ambiguity.
Dynamic 3D reconstruction of surgical scenes plays a critical role in robotic-assisted surgery. Gaussian Splatting (GS), while effective for novel view synthesis, struggles to recover accurate surface from a monocular view due to the implicit multi-Gaussian representation of the surface. Specifically, (1) the vertical overlap of Gaussians leads to floating artifacts, and (2) the random orientation of Gaussians affects the smoothness of the reconstructed surface. Consequently, these fragmented and misaligned Gaussians hinder downstream applications, e.g., geometry-aware endoscopic navigation and physics-integrated tissue mechanics simulation. To this end, we propose Endo-GSG, a unified framework that couples dynamic Gaussian splatting with an SDF field for dynamic surface-aware tissue reconstruction. To further enhance geometric fidelity, we design geometry-informed regularization losses that constrain Gaussian density and spatial positioning. The entire pipeline is jointly supervised by RGB images and predicted depth maps, enabling high-quality reconstruction and rendering even with sparse or monocular input. Experiments on public datasets demonstrate that Endo-GSG outperforms state-of-the-art methods in both rendering quality and geometric surface accuracy.
Recent works in Gaussian Splatting (GS) SLAM highlight the importance of geometric structure. However, existing methods often rely on either 2D or 3D Gaussian primitives, lacking the balance between geometry and appearance, thus failing to precisely model spatial structures. Furthermore, current frameworks either solely utilize SLAM to provide poses for GS or optimize poses and all Gaussian parameters indiscriminately, which neglect geometric consistency constraints essential for robust localization, creating a gap between GS and SLAM. In this paper, a tight-coupling SLAM system based on a Geometric Gaussian Field (GGF) is proposed. First, GGF combines explicit spatial structures with implicit neural residuals to adaptively regulate the Gaussian morphologies, seamlessly transitioning between 2D and 3D Gaussian primitives, ensuring precise spatial modeling. Second, a Geometric Consistency-Guided Refinement (GCGR) strategy is introduced, which exploits depth and normal maps rendered by GGF to construct critical constraints for accurate pose estimation. Simultaneously, updated poses and spatial information are fed back to refine Gaussian morphology, establishing mutual enhancement and tight-coupling between GS and SLAM. Extensive experiments demonstrate that GGF-SLAM outperforms state-of-the-art methods in tracking accuracy, photorealistic rendering, and geometric reconstruction.
PHOTONS: Pose-Free Human-Centric Photo-Realistic Real-Time Novel View Synthesis from Sparse Views
PDF ↗We present PHOTONS (Pose-Free Human-Centric Photo-Realistic Real-Time Novel View Synthesis from Sparse Views), a real-time framework for novel view synthesis without requiring camera calibration. Our method reconstructs consistent 3D Gaussian point clouds and synthesizes 2K photo-realistic novel views from arbitrary numbers (>=2) of freely placed cameras. PHOTONS faithfully renders dynamic human bodies amid complex backgrounds, including interactive object manipulation and fine-grained details (e.g., hair strands), while maintaining 25 FPS throughput on commodity GPU like NVIDIA RTX 4090. By combining pose-free spatial point cloud reconstruction with Gaussian parameter estimation, our method demonstrates strong resilience to occlusions and camera perturbations. Additionally, we develop a 3D stereo system that drastically reduces setup complexity compared to existing solutions. Experiments on public and custom datasets show that PHOTONS outperforms state-of-the-art methods in both efficiency and visual quality.
3D Gaussian splatting (3DGS) has recently demonstrated significant potential in computer vision, enabling high-fidelity 3D scene reconstruction with real-time rendering and fast training times. However, existing methods struggle in large, visually sparse, geometric self-similarity environments due to heavy reliance on image-based feature matching and depth information. In this work, we propose a novel reconstruction pipeline that reduces the dependence on visual features by incorporating IMU and LiDAR data to generate accurate point clouds and robustly localize images within the scene. Global colorization is achieved through 3D-to-2D projections of the localized images, which are then used to supervise 3DGS training. Our results demonstrate that the proposed pipeline significantly enhances the quality of 3D reconstruction for large, sparse scenarios, opening up new opportunities for applications in remote mapping and autonomous inspection.
HI-SLAM2: Geometry-Aware Gaussian SLAM for Fast Monocular Scene Reconstruction (Abstract Reprint)
PDF ↗We present HI-SLAM2, a geometry-aware Gaussian SLAM system that achieves fast and accurate monocular scene reconstruction using only RGB input. Existing Neural SLAM or 3DGS-based SLAM methods often trade off between rendering quality and geometry accuracy, our research demonstrates that both can be achieved simultaneously with RGB input alone. The key idea of our approach is to enhance the ability for geometry estimation by combining easy-to-obtain monocular priors with learning-based dense SLAM, and then using 3D Gaussian splatting as our core map representation to efficiently model the scene. Upon loop closure, our method ensures on-the-fly global consistency through efficient pose graph bundle adjustment and instant map updates by explicitly deforming the 3D Gaussian units based on anchored keyframe updates. Furthermore, we introduce a grid-based scale alignment strategy to maintain improved scale consistency in prior depths for finer depth details. Through extensive experiments on Replica, ScanNet, and ScanNet++, we demonstrate significant improvements over existing Neural SLAM methods and even surpass RGB-D-based methods in both reconstruction and rendering quality.
Splat-SAP: Feed-Forward Gaussian Splatting for Human-Centered Scene with Scale-Aware Point Map Reconstruction
PDF ↗We present Splat-SAP, a feed-forward approach to render novel views of human-centered scenes from binocular cameras with large sparsity. Gaussian Splatting has shown its promising potential in rendering tasks, but it typically necessitates per-scene optimization with dense input views. Although some recent approaches achieve feed-forward Gaussian Splatting rendering through geometry priors obtained by multi-view stereo, such approaches still require largely overlapped input views to establish the geometry prior. To bridge this gap, we leverage pixel-wise point map reconstruction to represent geometry which is robust to large sparsity for its independent view modeling. In general, we propose a two-stage learning strategy. In stage 1, we transform the point map into real space via an iterative affinity learning process, which facilitates camera control in the following. In stage 2, we project point maps of two input views onto the target view plane and refine such geometry via stereo matching. Furthermore, we anchor Gaussian primitives on this refined plane in order to render high-quality images. As a metric representation, the scale-aware point map in stage 1 is trained in a self-supervised manner without 3D supervision and stage 2 is supervised with photo-metric loss. We collect multi-view human-centered data and demonstrate that our method improves both the stability of point map reconstruction and the visual quality of free-viewpoint rendering.
Laboratory mice, particularly the C57BL/6 strain, are essential animal models in biomedical research. However, accurate 3D surface motion reconstruction of mice remains a significant challenge due to their complex non-rigid deformations, textureless fur-covered surfaces, and the lack of realistic 3D mesh models. Moreover, existing visual datasets for mice reconstruction only contain sparse viewpoints without 3D geometries. To fill the gap, we introduce MoReMouse, the first monocular dense 3D reconstruction network specifically designed for C57BL/6 mice. To achieve high-fidelity 3D reconstructions, we present three key innovations. First, we create the first high-fidelity, dense-view synthetic dataset for C57BL/6 mice by rendering a realistic, anatomically accurate Gaussian mouse avatar. Second, MoReMouse leverages a transformer-based feedforward architecture combined with triplane representation, enabling high-quality 3D surface generation from a single image, optimized for the intricacies of small animal morphology. Third, we propose geodesic-based continuous correspondence embeddings on the mouse surface, which serve as strong semantic priors, improving surface consistency and reconstruction stability, especially in highly dynamic regions like limbs and tail. Through extensive quantitative and qualitative evaluations, we demonstrate that MoReMouse significantly outperforms existing open-source methods in both accuracy and robustness.