论文检索

Deep Learning · Large Language Models

Fedor Rodionov, Abdelrahman Eldesokey, Michael Birsak, John Femiani, Bernard Ghanem, Peter Wonka

We introduce FloorplanQA, a diagnostic benchmark for evaluating spatial reasoning in large-language models (LLMs). FloorplanQA is grounded in structured representations of indoor scenes (e.g., kitchens, living rooms, bedrooms, bathrooms, and others), encoded symbolically in JSON or XML layouts. The benchmark covers core spatial tasks, including distance measurement, visibility, path finding, and object placement within constrained spaces. Our results across a variety of frontier open-source and commercial LLMs reveal that while models may succeed on shallow queries, they often fail to respect physical constraints and preserve spatial coherence, though they remain mostly robust to small spatial perturbations. FloorplanQA uncovers a blind spot in today’s LLMs: inconsistent reasoning about indoor layouts. We hope this benchmark inspires new work on language models that can accurately infer and manipulate spatial and geometric properties in practical settings.

Applications · Chemistry, Physics, and Earth Sciences

Jacopo Teneggi, SM Turzo, Tanya Marwah, Alberto Bietti, P. Douglas Renfrew, Vikram Mulligan, Siavash Golkar

Large language models (LLMs) are capable of emulating reasoning and using tools, creating opportunities for autonomous agents that execute complex scientific tasks. Protein design provides a natural testbed: although machine learning (ML) methods achieve strong results, these are largely restricted to canonical amino acids and narrow objectives, leaving unfilled need for a generalist tool for broad design pipelines. We introduce Agent Rosetta, an LLM agent paired with a structured environment for operating Rosetta, the leading physics-based heteropolymer design software, capable of modeling non-canonical building blocks and geometries. Agent Rosetta iteratively refines designs to achieve user-defined objectives, combining LLM reasoning with Rosetta's generality. We evaluate Agent Rosetta on design with canonical amino acids, matching specialized models and expert baselines, and with non-canonical residues-where ML approaches fail-achieving comparable performance. Critically, prompt engineering alone often fails to generate Rosetta actions, demonstrating that environment design is essential for integrating LLM agents with specialized software. Our results show that properly designed environments enable LLM agents to make scientific software accessible while matching specialized tools and human experts.

Applications · Robotics

Jiaxu Wang, Junhao He, Jingkai SUN, Yi Gu, Yunyang Mo, Jiahang Cao, Qiang Zhang, Renjing Xu

Learning real-world dynamics from visual observations is crucial for various domains. A common strategy is to calibrate simulators by estimating physical parameters, yet accuracy is ultimately bounded by the underlying physical models, which often assume materials are homogeneous and isotropic. Even if reasonable, real-world objects typically exhibit mild anisotropy and heterogeneity. After the near-isotropic backbone is well calibrated, these residual effects become the key bottleneck for further closing the real-to-sim gap. Although neural networks can fit dynamics end-to-end, such black-box modeling discards strong physical priors, leading to poor data efficiency and overfitting. Therefore, we propose MoSA, a motion-constrained stress adaptation framework that targets these residual effects to further improve real-to-sim dynamics learning. MoSA uses an isotropic model as a physics prior and learns residual stress operators to capture mild anisotropy and heterogeneity. It progressively adapts stresses via microplane-constrained redistribution in a physics-informed cascaded network. We further impose motion constraints by supervising temporal and spatial derivatives of the deformation field. Experimentally, our learned dynamics achieves superior accuracy, generalization, and robustness, while learning physically meaningful residual anisotropy. Finally, we validate MoSA in a robot manipulation setting, showing that better real-to-sim dynamics modeling translates into more reliable sim-to-real transfer.

Applications · Chemistry, Physics, and Earth Sciences

Liao Chang, Luotian Yuan, Yiping Ke, Ying Wei

Multi-step retrosynthesis planning is a fundamental challenge in organic chemistry, defined by its enormous search space. Existing methods typically formulate it as a Markov Decision Process (MDP) with a fixed choice of transition model (i.e., a single-step retrosynthesis model), and focus on improving *how to search* through better policies and value functions. However, *how the transition space itself is navigated* remains largely unexplored. This limitation is particularly urgent given our observation of pronounced *skill disparity* among single-step prediction models: different models exhibit substantially different performance across molecule states. Motivated by this observation, we introduce RetrOrchestrator, an LLM-powered agent that explicitly accounts for model skill disparity by reframing retrosynthesis planning as a Partially Observable Markov Decision Process (POMDP). By regarding each single-step prediction model as a tool, we further propose a scaffold-aware reinforcement learning algorithm to optimize navigation policy within the transition space. As a result, RetrOrchestrator jointly searches which molecule to expand and which single-step model to apply for the molecule at the current step. Empirically, RetrOrchestrator significantly outperforms static baselines on the Retro*-190 benchmark, achieving a state-of-the-art 94.21\% success rate as well as a Pareto front in both wallclock time and number of model queries.

Applications · Chemistry, Physics, and Earth Sciences

Mujie Lin, Yutian Liu, Yudi Guo, Yanzhen Hou, Yiheng Tao, Ruochong Zheng, Kaiwen Cheng, Xin Shan, Youdong Mao, Jie Chen

Generating long-horizon, all-atom molecular dynamics (MD) is difficult due to error accumulation in time-domain autoregressive models (causing drift) and fixed step-size constraints on temporal resolution. We propose **BioDynaSpec**, which reformulates protein dynamics as spatio-spectral generation: **Independent Windowed Fourier Decomposition (IWFD)** decomposes trajectories into independent windowed frequency representations, and a generator combines low-to-high frequency autoregression with diffusion denoising to reconstruct continuous motion. We introduce **Inter-Residue Frequency Coupling (IRFC)** bias, a learnable Gaussian distance bias in attention that injects a resonance-inspired structural prior to stabilize training and improve cross-residue, cross-frequency consistency. On ATLAS, BioDynaSpec improves 250-frame trajectory generation with $R_{250}=1.509$ Å (where $R_s$ denotes the mean per-frame C$\alpha$-RMSE over the first $s$ frames after alignment), reducing error by 60.4\% vs. MDGEN and 57.2\% vs. ProAR, and achieves the best PCA-2D displacement-profile correlation and stepwise distribution matching. For equilibrium conformational sampling, it achieves Root Mean $W_2=1.31$, MD PCA $W_2=0.90$, and Joint PCA $W_2=1.19$ (50.03\%, 35.25\%, and 47.58\% lower than the next best), while ablations show removing IRFC severely degrades RMSE/MAE and correlation.

Applications · Computer Vision

Zijie Lou, Xiangwei Feng, Jiaxin Wang, Jiangtao Yao, Fei Che, Tianbao Liu, WU CHENGJING, Xiaochao Qu, Luoqi Liu, Ting Liu

Existing video object removal methods predominantly rely on diffusion models following a noise-to-data paradigm, where generation starts from uninformative Gaussian noise. This approach discards the rich structural and contextual priors present in the original input video. Consequently, such methods often lack sufficient guidance, leading to incomplete object erasure or the synthesis of implausible content that conflicts with the scene's physical logic. In this paper, we reformulate video object removal as a video-to-video translation task via a stochastic bridge model. Unlike noise-initialized methods, our framework establishes a direct stochastic path from the source video (with objects) to the target video (objects removed). This bridge formulation effectively leverages the input video as a strong structural prior, guiding the model to perform precise removal while ensuring that the filled regions are logically consistent with the surrounding environment. To address the trade-off where strong bridge priors hinder the removal of large objects, we propose a novel adaptive mask modulation strategy. This mechanism dynamically modulates input embeddings based on mask characteristics, balancing background fidelity with generative flexibility. Extensive experiments demonstrate that our approach significantly outperforms existing methods in both visual quality and temporal consistency. The project page is https://bridgeremoval.github.io/.

Applications · Chemistry, Physics, and Earth Sciences

Shrimon Mukherjee, KISHALAY DAS, Partha Basuchowdhuri, Pawan Goyal, Niloy Ganguly

Fast and accurate prediction of crystal properties is a central challenge in new materials design. Graph Neural Networks and transformer-based models have emerged as powerful tools for this task due to their ability to encode the local structural environment of atoms within a crystal. However, these models are data hungry and in practice labeled data for crystal properties are very scarce. Pretrain–finetuning strategies, particularly those based on diffusion models, have shown promise in addressing these limitations. In this work, we introduce a novel latent-diffusion based pretraining framework CrysLDNet designed to mitigate the data scarcity issue. Our approach integrates a Variational Autoencoder (VAE) with a diffusion model during the pretraining stage. The VAE encoder maps 3D crystal structures into a smooth latent space, within which the diffusion process is applied. This latent diffusion pretraining enables the graph encoder to effectively capture structural and chemical semantics from large-scale unlabeled data, which can then be finetuned for specific property prediction tasks. Comprehensive experiments on popular DFT datasets for property prediction reveal that CrysLDNet significantly outperforms both training-from-scratch and pretrained baselines, with improvements of 4.26% and 4.90% on JARVIS and MP datasets. Additionally, the learned representations remain robust under sparse data conditions and are expressive enough to correct DFT errors when finetuned with limited experimental data.

Applications · Computer Vision

Weilun Feng, Guoxin Fan, Haotong Qin, Chuanguang Yang, Mingqiang Wu, Yuqi Li, Xiangqi Li, Zhulin An, Libo Huang, Dingrui Wang 等

Diffusion-based world models have shown strong potential for unified world simulation, but the iterative denoising remains too costly for interactive use and long-horizon rollouts. While feature caching can accelerate inference without training, we find that policies designed for single-modal diffusion transfer poorly to world models due to two world-model-specific obstacles: *token heterogeneity* from multi-modal coupling and spatial variation, and *non-uniform temporal dynamics* where a small set of hard tokens drives error growth, making uniform skipping either unstable or overly conservative. We propose **WorldCache**, a caching framework tailored to diffusion world models. We introduce *Curvature-guided Heterogeneous Token Prediction*, which uses a physics-grounded curvature score to estimate token predictability and applies a Hermite-guided damped predictor for chaotic tokens with abrupt direction changes. We also design *Chaotic-prioritized Adaptive Skipping*, which accumulates a curvature-normalized, dimensionless drift signal and recomputes only when bottleneck tokens begin to drift. Experiments on diffusion world models show that WorldCache delivers up to **3.7$\times$** end-to-end speedups while maintaining **98\%** rollout quality, demonstrating the vast advantages and practicality of WorldCache in resource-constrained scenarios.

Applications · Chemistry, Physics, and Earth Sciences

Zipeng Sun, Can Chen, Ye Yuan, Haolun Wu, Jiayao Gu, Christopher Pal, Xue Liu

We study offline black-box optimization (BBO), aiming to discover improved designs from an offline dataset of designs and labels, a problem common in robotics, DNA, and materials science with limited labeled samples. While recent work applies autoregressive LLMs to BBO by formatting tasks as natural-language prompts, their left-to-right design generation struggles to capture the strong bidirectional dependencies inherent in design problems. To address this, we propose adapting diffusion LLMs to offline BBO to leverage their bidirectional modeling capabilities. However, a domain gap exists between the natural text pre-training of diffusion LLMs and the heterogeneous signals in BBO (prompts, designs, and labels). To bridge this gap, we construct a unified prompt–response corpus and introduce delimiter tokens to explicitly mark field boundaries for *domain adaptation*. We further propose a two-stage *post-training* framework to align the diffusion LLM generation with high-label designs. The first stage performs supervised fine-tuning on the unified dataset via masked-response prediction, and the second stage adopts reinforcement learning with rewards defined by label improvements. Our method achieves state-of-the-art results on Design-Bench small-data settings. Code for our work is available here: https://anonymous.4open.science/r/Anonymous-dllm4bbo-D78A/README.md.

Deep Learning · Generative Models and Autoencoders

Xiaochao Deng, Jie Chen, Xiaogang Deng

Accurately modeling the full distributions of possible states is crucial for understanding statistical properties and enabling reliable predictions in complex, unsteady physical systems. Recently, diffusion models and flow matching have shown promise in these tasks. However, they remain limited in uncovering the general principles of systems from multiple short trajectories across condition space. In addition, they exhibit inferior adaptability to large irregular geometries, particularly in regions with sharp gradients. In this paper, we propose a condition-aware graph flow matching (CGFM) method that combines condition-aware flow matching with a hierarchical graph structure to learn the full distributions of physical systems from incomplete training data. Specifically, CGFM constructs a flow enabling smooth interpolation across physical conditions and parameterizes the graph-conditioned vector field through HieraGraphNet. HieraGraphNet performs message passing across multilevel graphs to capture multi-scale dynamics and facilitate long-range information interactions in physical systems. Moreover, we introduce a topology- and geometry-aware graph coarsening scheme that incorporates topological connectivity and local geometric density to construct reliable coarse graphs. We validate the effectiveness of CGFM on three canonical scenarios across both 2D and 3D dynamics, which demonstrate its superior performance compared with that of state-of-the-art baselines.

Deep Learning · Generative Models and Autoencoders

Andrew Millard, Fredrik Lindsten, Zheng Zhao

We introduce a guided stochastic sampling method that augments sampling from diffusion models with physics-based guidance derived from partial differential equation (PDE) residuals and observational constraints, ensuring generated samples remain physically admissible. We embed this sampling procedure within a new Sequential Monte Carlo (SMC) framework, yielding a scalable generative PDE solver. Across multiple benchmark PDE systems as well as multiphysics and interacting PDE systems, our method produces solution fields with lower numerical error than existing state-of-the-art generative methods.

Deep Learning · Generative Models and Autoencoders

Gyeonghoon Ko, Juho Lee

Riemannian diffusion models generalize score-based generative modeling to manifold-supported data via stochastic diffusion equations on the manifold. However, training requires sampling from and differentiating the manifold heat kernel, which is rarely available in closed form beyond a few highly symmetric manifolds. We propose a general approach that approximates the heat kernel by directly solving the manifold heat equation with a physics-informed neural network (PINN). Given an explicit manifold specification, we choose a coordinate system, derive the corresponding heat (Fokker--Planck) equation and a short-time asymptotic approximation, and then train a PINN to learn the log heat kernel. The resulting surrogate enables both forward noising (heat-kernel sampling) and conditional-score evaluation for denoising score matching. We demonstrate the method on diverse manifolds including $S^2$, $SO(3)$, $\mathrm{SPD}(n)$, and permutation-quotiented point clouds.

Applications · Time Series

Yangyou Liu, Zezhi Shao, Xinyu Chen, Hu Chen, Fei Wang, Yuankai Wu

Time series forecasting under non-stationarity faces a fundamental tension between capturing stable representations and adapting to distribution shifts. Existing methods implicitly rely on static historical assumptions, leading to a critical failure mode we term Phase Amnesia, where models become blind to the evolving global context. To resolve this, we formalize non-stationary dynamics through three physical hypotheses: Wold decomposition, dynamical phase evolution, and heteroscedastic manifold generation. These principles inspire PULSE, a physics-informed, plug-and-play framework adopting a Disentangle--Evolve--Simulate design philosophy. Specifically, PULSE utilizes phase-anchored disentanglement to resolve optimization interference caused by dominant trends, employs a Phase Router to actively generate future trajectories, and introduces Statistic-Aware Mixup (SAM) to ensure robustness against out-of-distribution volatility. Empirically, PULSE enables a simple MLP backbone to consistently outperform state-of-the-art Transformers across 12 real-world benchmarks. This validates that a correct physics-informed inductive bias is far more critical than raw architectural complexity for non-stationary forecasting.

Deep Learning · Generative Models and Autoencoders

Kijung Jeon, Michael Muehlebach, Molei Tao

Generative modeling within constrained sets is essential for scientific and engineering applications involving physical, geometric, or safety requirements (e.g., molecular generation, robotics). We present a unified framework for constrained diffusion models on generic nonconvex feasible sets $\Sigma$ that simultaneously enforces equality and inequality constraints throughout the diffusion process. Our framework incorporates both overdamped and underdamped dynamics for forward and backward sampling. A key algorithmic innovation is a computationally efficient landing mechanism that replaces costly and often ill-defined projections onto $\Sigma$, ensuring feasibility without iterative Newton solves or projection failures. By leveraging underdamped dynamics, we accelerate mixing toward the prior distribution, effectively alleviating the high simulation costs typically associated with constrained diffusion. Empirically, this approach reduces function evaluations and memory usage during both training and inference while preserving sample quality. On benchmarks featuring equality and mixed constraints, our method achieves comparable sample quality to state-of-the-art baselines while significantly reducing computational cost, providing a practical and scalable solution for diffusion on nonconvex feasible sets.

Applications · Time Series

Ruituo Wu, Hongyu Zhang, Qiang Wang, Jiawei Du, Wei Cui, Ce Zhu, Bing Li

Learning from continuous-time trajectories requires modeling multivariate sensor measurements generated by underlying physical or dynamical processes. Under extreme data compression and heterogeneous sampling, directly optimizing synthetic signals as discrete sample values becomes fundamentally misaligned with the underlining \emph{continuous-time physical processes}, often producing high-frequency, non-physical artifacts that overfit specific models and break reuse across architectures and sampling rates. We propose CHESS, a \emph{function-first} synthesis framework shifts optimization from discrete samples to underlying continuous-time signal trajectories. CHESS injects physics-induced structure by jointly enforcing low-rank spatial coherence and piecewise Chebyshev polynomial temporal parameterization, constraining synthesis to a physically meaningful function manifold. We provide theoretical analysis establishing explicit smoothness and stability guaranties. Experiments on diverse sensor testbeds under the dataset distillation protocol demonstrate CHESS consistently outperform state-of-the-art methods with a compression ratios up to $133\times$ for each synthetic sample. Furthermore, CHESS exhibits strong cross-architecture generalization and enables zero-shot adaptation across different sampling resolutions.

Applications · Health / Medicine

Yinbo Liu, Keyang Ye, Wenshan Sun, Handi Gao, Tian Tian

Reconstructing unified continuous dynamics from sparse, non-contiguous, and unpaired point cloud snapshots remains a fundamental challenge in spatiotemporal analysis for computer vision and developmental biology. Existing methods, including scene flow and Optimal Transport-based approaches, are limited either by explicit reliance on point-wise correspondences or by cumulative errors from frame-to-frame propagation and temporal discontinuity, and limited ability to model multi-attribute dynamics such as gene expression and population changes. We propose FlowCloud, a variational Neural Ordinary Differential Equation (Neural ODE) generative framework. FlowCloud aggregates information from all observed time points into a joint latent representation that initializes a Neural ODE $z(t)$ enabling continuous spatiotemporal evolution modeling while mitigating propagation-induced errors and preserving temporal consistency. Training is performed without predefined correspondences using a multi-faceted objective with complementary roles: Sinkhorn distance for global distribution alignment, Chamfer distance for local geometric consistency, trajectory regularization to encourage smooth and physically plausible dynamics, and supervised losses for multi-attribute prediction. Experiments on human motion and developmental biology datasets demonstrate improved interpolation accuracy and promising short-term extrapolation performance. By unifying geometry, attributes, and population dynamics within a continuous latent framework, FlowCloud offers a novel and robust solution for continuous dynamic reconstruction from unstructured spatiotemporal observations.

Applications · Chemistry, Physics, and Earth Sciences

Weilong Chen, Bojun Zhao, Jan Eckwert, Julija Zavadlav

Sampling equilibrium molecular configurations from the Boltzmann distribution is a longstanding challenge. Boltzmann Generators (BGs) address this by combining exact-likelihood generative models with importance sampling, but their practical scalability is limited. Meanwhile, coarse-grained surrogates enable the modeling of larger systems by reducing effective dimensionality, yet often lack the reweighting process required to ensure asymptotically correct statistics. In this work, we propose Coarse-Grained Boltzmann Generators (CG-BGs), a principled framework that unifies scalable reduced-order modeling with the exactness of importance sampling. CG-BGs act in a coarse-grained coordinate space, using a learned potential of mean force (PMF) to reweight samples generated by a flow-based model. Crucially, we show that this PMF can be efficiently learned from rapidly converged data via force matching. Our results demonstrate that CG-BGs faithfully capture complex interactions mediated by explicit solvent within highly reduced representations, establishing a scalable pathway for the unbiased sampling of larger molecular systems.

Applications · Chemistry, Physics, and Earth Sciences

Xin Ju, Hadrian Fung, Yuyan Zhang, Carl Jacquemyn, Matthew Jackson, Randolph Settgast, Sally Benson, Gege Wen

The Earth's subsurface is a cornerstone of modern society, providing essential energy resources like hydrocarbons, geothermal, and minerals while serving as the primary reservoir for $CO_2$ sequestration. However, full physics numerical simulations of these systems are notoriously computationally expensive due to geological heterogeneity, high resolution requirements, and the tight coupling of physical processes with distinct propagation time scales. Here we propose the Adaptive Physics Transformer (APT), a geometry-, mesh-, and physics-agnostic neural operator that explicitly addresses these challenges. APT fuses a graph-based encoder to extract high-resolution local heterogeneous features with a global attention mechanism to resolve long-range physical impacts. Our results demonstrate that APT outperforms state-of-the-art architectures in subsurface tasks across both regular and irregular grids with robust super-resolution capabilities. Notably, APT is the first architecture that directly learns from adaptive mesh refinement simulations. We also demonstrate APT's capability for cross-dataset learning, positioning it as a robust and scalable backbone for large-scale subsurface foundation model development.

Applications · Computer Vision

Jihyun Lee, Changmin Lee, Donghwan Kim, Tae-Kyun Kim

While existing methods for reconstructing hand–object interactions have made impressive progress, they either focus on rigid or part-wise rigid objects—limiting their ability to model real-world objects (e.g., cloth, stuffed animals) that exhibit highly non-rigid deformations—or model deformable objects without full 3D hand reconstruction. To bridge this gap, we present PhysHandi (Physics-based Reconstruction of Hand and Deformable Object Interactions), a framework that enables full 3D reconstruction of both interacting hands and non-rigid objects. Our key idea is to physically simulate object deformations driven by forces induced from densely reconstructed 3D hand motions, ensuring that the reconstructed object dynamics are both physically plausible and coherent with the interacting hand movements. Furthermore, we demonstrate that such simulation of object deformations can, in turn, refine and improve hand reconstruction via inverse physics. In experiments, PhysHandi outperforms the state-of-the-art baseline across reconstruction and future prediction.

Deep Learning · Generative Models and Autoencoders

Yuanbin Man, Ying Huang, Zhile Ren, Miao Yin

4D generation, or dynamic 3D content generation, integrates spatial, temporal, and view dimensions to model realistic dynamic scenes, playing a foundational role in advancing world models and physical AI. However, maintaining long-chain consistency across both frames and viewpoints through the unique spatio-camera-motion (SCM) attention mechanism introduces substantial computational and memory overhead, often leading to out-of-memory (OOM) failures and prohibitive generation times. To address these challenges, we propose Turbo4DGen, an ultra-fast acceleration framework for diffusion-based multi-view 4D content generation. Turbo4DGen introduces a spatiotemporal cache mechanism that persistently reuses intermediate attention across denoising steps, combined with dynamically semantic-aware attention pruning and an adaptive SCM chain bypass scheduler, to drastically reduce redundant SCM attention computation. Our experimental results show that Turbo4DGen achieves an average 9.7$\times$ speedup without quality degradation on the ObjaverseDy and Consistent4D datasets. To the best of our knowledge, Turbo4DGen is the first dedicated acceleration framework for 4D generation.