Differentiable volume rendering enables gradient-based optimization of volumetric scenes, but unbiased estimators suffer from high gradient variance. We observe that the extinction gradients split into two components on structurally different integration domains: a scattering term evaluated at a single path vertex, and a transmittance term integrated along the ray segment. Because the domains are mismatched, existing estimators sample the two components at different locations, leaving the negative correlation between their opposite-signed contributions unexploited. We expose this overlooked correlation and exploit it through a principle we call sample matching: evaluate both components at shared sample locations. To enable this, we derive the first reformulation of the differential path integral that couples the two contributions within a single integrand, yielding an unbiased Monte Carlo estimator that ties them together by construction. For efficiency, the estimator reuses partially sampled light paths and amortizes in-scattering cost by evaluating gradients at multiple probe points per segment. On voxel-grid reconstruction, our estimator reduces gradient variance by up to 80% over differential ratio tracking (DRT), yielding faster convergence and higher reconstruction quality.
论文检索
输入标题、作者或关键词，从 5,999 篇学术成果中精准定位
This paper describes a strategy for vectorizing 3D scenes with proper occlusion. Given a collection of curves derived from 3D geometry (silhouettes, isolines, material boundaries, etc.), we produce a 2D vector image that partitions the image plane into a planar map of solid shaded regions. The method is agnostic to surface representation, handling for instance curves obtained from polygonal, NURBS or subdivision surfaces. The output likewise supports curve segments of arbitrary parametric type. Our key observation is that the spatial hierarchy used to accelerate curve-curve intersections provides the fundamental representation of the planar map itself. This approach provides geometric flexibility, since general curve-curve intersection problems are replaced with simpler curve-line intersection. Simultaneously, it provides robustness since cells of the spatial hierarchy define a well-defined planar map, even in the presence of numerical errors. For instance, it automatically handles "curve soup" where segment endpoints are not explicitly connected in the input file. The method scales to a large number of primitives, and is output sensitive: it resolves intersections only up to a user-defined precision—while still providing topologically valid output. We evaluate the method on a collection of challenging tests, showing that it is both more robust and orders of magnitude more efficient than existing curve arrangement techniques, such as those found in CGAL.
Recent advancements in autoregressive transformers have demonstrated remarkable potential for generating artist-quality meshes. However, the token ordering strategies employed by existing methods typically fail to meet professional artist standards, where coordinate-based sorting yields inefficiently long sequences, and patch-based heuristics disrupt the continuous edge flow and structural regularity essential for high-quality modeling. To address these limitations, we propose Strips as Tokens (SATO), a novel framework with a token ordering strategy inspired by triangle strips. By constructing the sequence as a connected chain of faces that explicitly encodes UV boundaries, our method naturally preserves the organized edge flow and semantic layout characteristic of artist-created meshes. A key advantage of this formulation is its unified representation, enabling the same token sequence to be decoded into either a triangle or quadrilateral mesh. This flexibility facilitates joint training on both data types: large-scale triangle data provides fundamental structural priors, while high-quality quad data enhances the geometric regularity of the outputs. Extensive experiments demonstrate that SATO consistently outperforms prior methods in terms of geometric quality, structural coherence, and UV segmentation.
Despite recent advancements in native latent diffusion models for single-view 3D generation, they still suffer from inadequate geometric details due to the global entanglement of VecSet-based representations and inaccurate structural topology arising from single-view ambiguity. Existing solutions often rely on model scaling over large-scale datasets to improve fidelity, which is computationally prohibitive and inefficient in capturing fine-grained details. In this work, we propose Tempo3D, a resource-efficient paradigm for high-fidelity VecSet-based 3D generation via efficient temporal-aware fine-tuning and multi-view latent aggregation. Our approach consists of two critical components: (1) To facilitate robust detail learning, we establish the TempoDetail dataset and design the Time-Segmented LoRA (TS-LoRA) module. TS-LoRA exploits the inherent "structure-to-details" temporal transition of flow-based models to decouple feature learning, significantly bolstering the capacity to produce high-frequency details without large-scale retraining. (2) To mitigate geometric inaccuracies stemming from single-view ambiguity, we propose a multi-view latent aggregation strategy incorporating Soft Flow Trajectory Projection (Soft FTP) and Score-based Velocity Field Aggregation (SVFA). By aggregating reference views derived from Multimodal Large Language Models (MLLMs), these components resolve the intrinsic conflicts between velocity vectors caused by cross-view latent discrepancies, thereby preventing geometric collapse and ensuring directional consistency. Moreover, the integration of MLLMs empowers Tempo3D to support precise semantic 3D editing via natural language prompts while maintaining structural integrity. Experiments demonstrate that Tempo3D outperforms state-of-the-art baselines in generation quality, geometric precision, and editability under a highly efficient computational budget.
We introduce SegviGen, a framework that repurposes native 3D generative models for 3D part segmentation. Existing pipelines either lift strong 2D priors into 3D via distillation or multi-view mask aggregation, often suffering from cross-view inconsistency and blurred boundaries, or explore native 3D discriminative segmentation, which typically requires large-scale annotated 3D data and substantial training resources. In contrast, SegviGen leverages the structured priors encoded in pretrained 3D generative model to induce segmentation through distinctive part colorization, establishing a novel and efficient framework for part segmentation. Specifically, SegviGen encodes an input 3D asset and predicts part-indicative colors on active voxels of a geometry-aligned reconstruction. It supports interactive part segmentation, full segmentation, and full segmentation with 2D guidance in a unified framework. Extensive experiments show that SegviGen improves over the prior state of the art by 40% on interactive part segmentation and by 15% on full segmentation, while using only 0.32% of the training data. This undoubtedly demonstrates that pretrained 3D generative priors transfer effectively to 3D part segmentation, enabling strong performance with limited supervision. Code and pretrained weights are publicly available at https://github.com/Nelipot-Lee/SegviGen.
Selection is a core operation in interactive image editing, enabling tasks such as composition or manipulation. To be practically useful, a user should be able to specify and disambiguate the desired selection region through either text- or click-based interactions, and the system should support selecting not only objects but also other criteria, such as materials. Material-based selection can be particularly valuable for tasks like re-texturing surfaces or consistently editing all instances of a specific material in a scene. However, existing vision–language-model (VLM) based selection methods are largely object-centric and typically support only a single interaction modality, limiting their applicability in real editing workflows. In this work, we thus present Mask Any Object And Material (MAOAM), a unified selection framework that enables precise object- and material-level selection across both text- and click-based interactions. MAOAM leverages a VLM with a segmentation head to produce pixel-accurate masks from user prompts: the VLM interprets the user’s selection intent — object- or material-level — and encodes visual entities, attributes, and spatial relations, while the segmentation head decodes the VLM’s output token into a mask. A key challenge is that material selection datasets with text annotations are unavailable. We therefore propose a scalable data generation pipeline: we collect real and synthetic images with material masks, then leverage VLMs to generate material descriptions with rich visual-semantic information. Using the generated data, we train MAOAM with a multi-task objective over click- and text-based selection, along with an auxiliary VQA task derived from the material descriptions to facilitate deeper material understanding. Despite being trained with uni-modal prompts, our model exhibits an emergent improvement in selection quality when combining text and clicks at inference time, enabling more flexible image editing workflows. Experiments demonstrate accurate and coherent selections across diverse objects, materials, and interaction scenarios, highlighting robustness in practice.
We present a global, discrete optimization framework for quad-dominant mesh reduction that formulates simplification as an edge selection problem under coupled topological and geometric constraints. Unlike greedy edge-collapse heuristics, which make irreversible local decisions, and rigid poly-chord removal schemes, which over-constrain global structure, our method casts quad-dominant reduction as an Integer Linear Programming (ILP) problem that jointly optimizes edge flow continuity and geometric fidelity in a single global optimization. To make this formulation both tractable and adaptive, we segment poly-chords at geometrically meaningful breakpoints to relax otherwise rigid topological constraints, and introduce geometry-aware soft constraints that steer the optimization toward preserving high-curvature and deformation-critical regions. Extensive experiments demonstrate that our approach consistently outperforms state-of-the-art methods in preserving quad structure and edge flow, while achieving competitive geometric accuracy and significantly benefiting downstream editing applications.
This work introduces FilmGPT, an autoregressive transformer designed to address the challenge of video montage – turning a collection of raw, “unwatchable” footage into coherent cinematic sequences. Inspired by language learning in modern LLMs, we train a long-context autoregressive transformer on a large corpus of movies. The aim is to implicitly capture the “grammar” of film directly from data rather than from hand-coded rules. Unlike other generative models, FilmGPT does not generate any new video frames. Instead, at inference time, we introduce a footage-constrained decoding algorithm to select the best next shot from the input raw footage according to the statistical patterns learned from films. We first evaluate these learned statistics directly by using the FilmGPT autoregressive model for next shot prediction on a standard benchmark of shot sequence ordering, outperforming the previous state of the art. We then evaluate our footage-constrained decoding algorithm on the full film editing task via a user study, and find that our FilmGPT-based editing significantly outperforms previous approaches. Finally, we demonstrate the applicability of FilmGPT to a wide range of applications in video montage, from automatic video segment trimming to human-in-the-loop film editing. Please see our supplementary material for qualitative results.
Time-varying scene meshes are a widely used representation for volumetric video, offering immersive six degrees of freedom (6DoF) interaction in virtual environments. However, their significantly larger size presents major challenges for efficient storage, Internet transmission, and real-time streaming. Existing mesh compression standards are not well-suited for complex scene meshes with dynamic content, and no standardized encoding techniques have been widely adopted for this class of data. To address this gap, we propose TSMC, a novel compression framework for 3D time-varying scene mesh sequences. TSMC uses voxel-based analysis to establish temporally stable correspondences, segment static and dynamic regions, and extract volume-tracked reference meshes for dynamic content to support inter-frame prediction. Static backgrounds and reference meshes are compressed using Google Draco, while dynamic displacements are encoded using a combination of the Karhunen-Loève Transform (KLT) and Laplacian coordinates. Extensive experiments demonstrate that at 30 fps and an SSIM of 0.9, TSMC reduces bandwidth usage and decoding time by up to 56.54% and 85.42%, respectively, compared to state-of-the-art mesh compression methods. Our code and dataset are available at https://github.com/SINRG-Lab/TSMC.
We present an empirical study of free-view video compression for dynamic scenes reconstructed with 3D Gaussian Splatting, examining how practical pipeline design choices affect reconstruction fidelity and storage efficiency. Rather than introducing new representations, we analyze how commonly used components, including temporal chunking, deformation-based reconstruction, and quantization-aware training, interact in practice. We observe that partitioning long sequences into shorter temporal segments, such as GOPs, simplifies optimization and improves reconstruction fidelity, but can introduce additional storage overhead. We further show that encouraging smooth motion vectors across both space and time produces deformation signals that are easier for standard video codecs to compress, leading to improved rate–distortion performance. When integrated into a unified pipeline, these design choices consistently benefit different deformation-based reconstruction methods. Across multiple datasets, our approach achieves 20% storage reduction compared with state-of-the-art methods while preserving or improving visual quality, and we discuss sources of variability and ambiguity in current training and evaluation protocols.
Monocular video–based human mesh recovery (HMR) has made significant progress in recent years, yet existing methods often fail to reconstruct physically plausible motion during highly dynamic airborne movements such as jumping or acrobatics. These failure cases arise from motion blur, rapid orientation changes, and the lack of suitable training data, leading to temporally inconsistent and physically implausible results. We propose a novel method for reconstructing 3D airborne motion by refining inaccurate estimates produced by state-of-the-art HMR systems. Our approach extracts key physical quantities, identifies reliable motion segments based on physical consistency, and connects them using a homotopy-aware trajectory optimization. A global angular momentum constraint is then enforced over the entire motion, and global motion and local poses are jointly optimized under physical and temporal smoothness constraints. Experiments on challenging in-the-wild videos demonstrate that our method produces more physically consistent and temporally coherent airborne motions than existing refinement approaches.
Boundary Representation (B-rep) is the most commonly used data format in Computer-Aided Design (CAD) due to its analytical precision and direct support for parametric editing. However, its heterogeneous data structure— continuous parametric geometry with discrete topological graphs—poses fundamental challenges for deep learning models. Existing methods often directly predict the heterogeneous B-rep graph, relying on fixed-size padding or sequential tokenization to handle the varying cardinality of the geometric primitives. These approaches struggle with the combinatorial complexity of CAD models. The discrete, non-differentiable nature of the graph data structure prevents end-to-end optimization of the geometry and watertightness. In this work we introduce DualBrep, a novel continuous representation that unifies B-rep geometry and topology within a fully structured Euclidean domain. DualBrep encodes a CAD model using dual scalar fields: a Signed Distance Function (SDF) to represent the global shape geometry, and an Unsigned Distance Field (UDF) that implicitly encodes the topological structure via a Voronoi partitioning of the surface elements. Rather than processing these fields independently, we compress them into a single latent space. While the dual-field formulation alone already gives reconstruction a flexible, primitive-free segmentation signal that adapts to arbitrary face counts and surface types, the shared latent also makes generation tractable: a Flow Matching model can sample geometry and topology jointly from a single code, avoiding the error accumulation that plagues sequential or autoregressive B-rep predictors. Finally, we use a neural rebuilder to extract explicit B-rep models—comprising both prismatic and free-form primitives—directly from our continuous dual scalar fields. We demonstrate that DualBrep serves as a robust, unified backbone for CAD B-rep learning, achieving strong performance in both reverse engineering from raw point clouds and generative modeling via latent flow matching. Code is available at https://github.com/AutodeskAILab/DualBrep.
In remote or underserved regions, where road networks are often either unavailable or poorly mapped, and GPS signals are sparse and unreliable, the quality of individual trajectories is severely compromised. In such contexts, web crowdsourced data becomes essential for accurately recovering trajectories and compensating for missing spatial information in the absence of explicit road networks. However, this situation introduces two key challenges:(i) scarcity of data in unseen regions, restricting transfer learning; and (ii) the necessity to infer latent movement structures under roadless conditions. To address these, we propose Region-aware Hierarchical Trajectory Recovery (RHTR) model, designed for location inference from web crowdsourced data in sparse, roadless scenarios. RHTR constructs multi-scale implicit grid-based offset maps from historical location data, with coarse grids capturing global patterns and fine grids refining local details. These sequential representations form region-aware encodings by sampling information around observed points, facilitating trajectory recovery. A coarse-to-fine mechanism leverages contextual information to progressively reconstruct missing segments. Experiments on two public datasets—simulating underserved and disaster-like settings with cross-region transfer—demonstrate that RHTR achieves state-of-the-art performance in trajectory recovery.
The rapid growth of video content on platforms such as TikTok and YouTube has intensified the spread of multimodal hate speech, where harmful cues emerge subtly and asynchronously across visual, acoustic, and textual streams. Existing research primarily focuses on video-level classification, leaving the practically crucial task of temporal localisation, identifying when hateful segments occur, largely unaddressed. This challenge is even more noticeable under weak supervision, where only video-level labels are available, and static fusion or classification-based architectures struggle to capture cross-modal and temporal dynamics. To address these challenges, we propose MultiHateLoc, the first framework designed for weakly-supervised multimodal hate localisation. MultiHateLoc incorporates (1) modality-aware temporal encoders to model heterogeneous sequential patterns, including a tailored text-based preprocessing module for feature enhancement; (2) dynamic cross-modal fusion to adaptively emphasise the most informative modality at each moment and a cross- modal contrastive alignment strategy to enhance multimodal feature consistency; (3) a modality-aware MIL objective to identify discriminative segments under video-level supervision. Despite relying solely on coarse labels, MultiHateLoc produces fine-grained, interpretable frame-level predictions. Experiments on HateMM and MultiHateClip show that our method achieves state-of-the-art performance in the localisation task. Code is available at https://github.com/Multimodal-Intelligence-Lab-MIL/MultiHateLoc.
Accurate semantic segmentation is crucial for rapid risk and severity assessment following flash flood disasters. However, in drone networks deployed for disaster recovery, bandwidth constraints mean the backhaul transmission of massive imagery required for centralized processing is highly prone to causing network congestion. Federated Learning, a decentralized collaborative paradigm that permits drones to process data locally, offers an ideal pathway to address this challenge. Nonetheless, existing FL methods are plagued by the dilemma of excessive communication overhead and insufficient segmentation performance. Therefore, this paper proposes a Communication-Efficient Federated Distillation (CEFD) framework. The core of this framework lies in the design of a lightweight, multi-level knowledge representation. It discards the transmission of bulky parameters, opting instead for the efficient exchange of high-level semantic logits and key intermediate features that have undergone feature-space dimensionality reduction. On the server side, adaptive weighted aggregation is then utilized to construct a robust global knowledge model. Experimental results show that CEFD achieved high segmentation performance on the LSD dataset, attaining an mIoU of 0.6226, a Mean Recall of 0.6532, an Accuracy of 0.9794, and an MAE of 0.0400, while reducing communication overhead by approximately 99.5%. This enables scalable collaborative intelligence for resource-constrained edge devices, a key capability for the future web systems.
Live streaming platforms require real-time monitoring and reaction to social signals, utilizing partial and asynchronous evidence from video, text, and audio. We propose StreamSense, a streaming detector that couples a lightweight streaming encoder with selective routing to a Vision–Language Model (VLM) expert. StreamSense handles most timestamps with the lightweight streaming encoder, escalates hard/ambiguous cases to the VLM, and defers decisions when context is insufficient. The encoder is trained using (i) a cross-modal contrastive term to align visual/audio cues with textual signals, and (ii) an IoU-weighted loss that down-weights poorly overlapping target segments, mitigating label interference across segment boundaries. We evaluate StreamSense on multiple social streaming detection tasks (e.g., sentiment classification and hate content moderation), and the results show that StreamSense achieves higher accuracy than VLM-only streaming while only occasionally invoking the VLM, thereby reducing average latency and compute. Our results indicate that selective escalation and deferral are effective primitives for understanding streaming social tasks. Code is publicly available on GitHub.
User behavior on social media---from scrolling and viewing to liking, reposting, and posting---yet most research relies on self-reports that obscure fine-grained usage patterns. We analyze high-resolution activity logs from 209 U.S. X (Twitter) users tracked over four weeks to identify distinct behavioral profiles based on session-level features. Latent profile analysis reveals three groups---interactors (32.52%), lurkers (60.45%), and creators (7.03%) that differ in engagement intensity, demographics, and content exposure. Interactors and lurkers skew younger and Democratic, whereas creators skew older and more Republican, consuming more ideological and low-credibility content. These results link behavioral heterogeneity to systematically different information environments and suggest that platform interventions may operate unevenly across user types. Our findings demonstrate the value of log-based behavioral segmentation for understanding online participation and motivate profile-aware platform governance and content moderation strategies.
In this paper, we propose Pr ecision-Informed Semantic Modeling (PRISM), a structured topic modeling framework combining the benefits of rich representations captured by LLMs with the low cost and interpretability of latent semantic clustering methods. PRISM fine-tunes a sentence encoding model using a sparse set of LLM-provided labels on samples drawn from some corpus of interest. We segment this embedding space with thresholded clustering, yielding clusters that separate closely related topics within some narrow domain. Across multiple corpora, PRISM improves topic separability over state-of-the-art local topic models and even over clustering on large, frontier embedding models while requiring only a small number of LLM queries to train. This work contributes to several research streams by providing (i) a student-teacher pipeline to distill sparse LLM supervision into a lightweight model for topic discovery; (ii) an analysis of the efficacy of sampling strategies to improve local geometry for cluster separability; and (iii)~an effective approach for web-scale text analysis, enabling researchers and practitioners to track nuanced claims and subtopics online with an interpretable, locally deployable framework.
Behavioral patterns captured in embeddings learned from interaction data are pivotal across various stages of production recommender systems. However, in the initial retrieval stage, practitioners face an inherent tradeoff between embedding expressiveness and the scalability and latency of serving components, resulting in the need for representations that are both compact and expressive. To address this challenge, we propose a training strategy for learning high-dimensional sparse embedding layers in place of conventional dense ones, balancing efficiency, representational expressiveness, and interpretability. To demonstrate our approach, we modified the production-grade collaborative filtering autoencoder ELSA, achieving up to 10× reduction in embedding size with no loss of recommendation accuracy, and up to 100× reduction with only a 2.5% loss. Moreover, the active embedding dimensions reveal an interpretable inverted-index structure that segments items in a way directly aligned with the model's latent space, thereby enabling integration of segment-level recommendation functionality (e.g., 2D homepage layouts) within the candidate retrieval model itself. Source codes, additional results, as well as a live demo are available at https://github.com/zombak79/compressed\_elsa.
In recent years, the success of large language models (LLMs) has driven the exploration of scaling laws in recommender systems. However, models that demonstrate scaling laws are actually challenging to deploy in industrial settings for modeling long sequences of user behaviors, due to the high computational complexity of the standard self-attention mechanism. Despite various sparse self-attention mechanisms proposed in other fields, they are not fully suited for recommendation scenarios. This is because user behaviors exhibit personalization and temporal characteristics: different users have distinct behavior patterns, and these patterns change over time, with data from these users differing significantly from data in other fields in terms of distribution. To address these challenges, we propose SparseCTR, an efficient and effective model specifically designed for long-term behaviors of users. To be precise, we first segment behavior sequences into chunks in a personalized manner to avoid separating continuous behaviors and enable parallel processing of sequences. Based on these chunks, we propose a three-branch sparse self-attention mechanism to jointly identify users' global interests, interest transitions, and short-term interests. Furthermore, we design a composite relative temporal encoding via learnable, head-specific bias coefficients, better capturing sequential and periodic relationships among user behaviors. Extensive experimental results show that SparseCTR not only improves efficiency but also outperforms state-of-the-art methods. More importantly, it exhibits an obvious scaling law phenomenon, maintaining performance improvements across three orders of magnitude in FLOPs. In online A/B testing, SparseCTR increased CTR by 1.72% and CPM by 1.41%. Our source code is available at https://github.com/laiweijiang/SparseCTR.