论文检索

Computer Vision · Image and Video Generation

Jianhao Yuan, Fabio Pizzati, Francesco Pinto, Lars Kunze, Ivan Laptev, Paul Newman, Philip Torr, Daniele De Martini

Intuitive physics understanding in video diffusion models plays an essential role in building general-purpose physically plausible world simulators, yet accurately evaluating such capacity remains a challenging task due to the difficulty in disentangling physics correctness from visual appearance in generation. To the end, we introduce LikePhys, a training-free method that evaluates intuitive physics in video diffusion models by distinguishing physically valid and impossible videos using the denoising objective as an ELBO-based likelihood surrogate on a curated dataset of valid-invalid pairs. By testing on our constructed benchmark of twelve scenarios spanning over four physics domains, we show that our evaluation metric, Plausibility Preference Error (PPE), demonstrates strong alignment with human preference, outperforming state-of-the-art evaluator baselines. We then systematically benchmark intuitive physics understanding in current video diffusion models. Our study further analyses how model design and inference settings affect intuitive physics understanding and highlights domain-specific capacity variations across physical laws. Empirical results show that, despite current models struggling with complex and chaotic dynamics, there is a clear trend of improvement in physics understanding as model capacity and inference settings scale up.

Computer Vision · Vision Models & Multimodal

Anbu Huang

This blogpost develops a geometric and probabilistic lens on diffusion priors for inverse problems. We show that a wide range of methods mostly instantiate two operator-splitting paradigms, i.e., posterior-guided sampling and clean-space local-MAP optimization. Through manifold diagrams, Tweedie-based animations, and step-by-step derivations, we explain how these paradigms decouple a pretrained diffusion prior from measurement physics, clarify when they approximate full posterior sampling versus MAP estimation, and distill practical design rules for building robust diffusion-based inverse solvers.

Applications · Physics

Vladimir Fanaskov, Ivan Oseledets

Discretisation invariance, a recent innovation in scientific machine learning, is a requirement that ensures an architecture can process inputs of different resolutions. In this post, we formally define this property, provide examples, generate datasets, train architectures, and discuss whether discretisation invariance is living up to its promise.

Applications · Physics

Yingjie Tan, Quanming Yao, Yaqing Wang

Spatiotemporal partial differential equations (PDEs) underpin a wide range of scientific and engineering applications. Neural PDE solvers offer a promising alternative to classical numerical methods. However, existing approaches typically require large numbers of training trajectories, while high-fidelity PDE data are expensive to generate. Under limited data, their performance degrades substantially, highlighting their low data efficiency. A key reason is that PDE dynamics embody strong structural inductive biases that are not explicitly encoded in neural architectures, forcing models to learn fundamental physical structure from data. A particularly salient manifestation of this inefficiency is poor generalization to unseen source terms. In this work, we revisit Green’s function theory—a cornerstone of PDE theory—as a principled source of structural inductive bias for PDE learning. Based on this insight, we propose DGNet, a discrete Green network for data-efficient learning of spatiotemporal PDEs. The key idea is to transform the Green’s function into a graph-based discrete formulation, and embed the superposition principle into the hybrid physics–neural architecture which reduces the burden of learning physical priors from data, thereby improving sample efficiency. Across diverse spatiotemporal PDE scenarios, DGNet consistently achieves state-of-the-art accuracy using only tens of training trajectories. Moreover, it exhibits robust zero-shot generalization to unseen source terms, serving as a stress test that highlights its data-efficient structural design.

Applications · Physics

Louis Claeys, Artur Goldman, Zebang Shen, Niao He

High-dimensional stochastic optimal control (SOC) becomes harder with longer planning horizons: existing methods scale linearly in the horizon $T$, with performance often deteriorating exponentially. We overcome these limitations for a subclass of linearly-solvable SOC problems—those whose uncontrolled drift is the gradient of a potential. In this setting, the Hamilton-Jacobi-Bellman equation reduces to a linear PDE governed by an operator $\mathcal{L}$. We prove that, under the gradient drift assumption, $\mathcal{L}$ is unitarily equivalent to a Schrödinger operator $\mathcal{S} = -\Delta + \mathcal{V}$ with purely discrete spectrum, allowing the long-horizon control to be efficiently described via the eigensystem of $\mathcal{L}$. This connection provides two key results: first, for a symmetric linear-quadratic regulator (LQR), $\mathcal{S}$ matches the Hamiltonian of a quantum harmonic oscillator, whose closed-form eigensystem yields an analytic solution to the symmetric LQR with arbitrary terminal cost. Second, in a more general setting, we learn the eigensystem of $\mathcal{L}$ using neural networks. We identify implicit reweighting issues with existing eigenfunction learning losses that degrade performance in control tasks, and propose a novel loss function to mitigate this. We evaluate our method on several long-horizon benchmarks, achieving an order-of-magnitude improvement in control accuracy compared to state-of-the-art methods, while reducing memory usage and runtime complexity from $\mathcal{O}(Td)$ to $\mathcal{O}(d)$.

Applications · Physics

Haining Pan, James Roggeveen, Erez Berg, Juan Alvarez, Debanjan Chowdhury, Surya Ganguli, Federico Ghimenti, Juraj Hasik, Henry Hunt, Hong-Chen Jiang 等

Large language models (LLMs) have demonstrated remarkable progress in coding and mathematical problem-solving; however, evaluation on advanced research-level problems in the hard sciences remains scarce. To fill this gap, we present \cmt, a dataset of 50 original problems covering condensed matter theory (CMT) at the level of an expert researcher. The solution for these problems involve analytical and computational approaches commonly used in quantum many-body physics and classical statistical mechanics. The dataset has been designed and verified by a worldwide panel of expert researchers through a collaborative environment. Topics in the dataset include Hartree-Fock mean-field theory, exact diagonalization methods, quantum Monte Carlo sampling, density matrix renormalization group, quantum statistical mechanics, classical statistical mechanics, and model building. We evaluate different LLMs by programmatically checking LLM-generated solutions against expert-supplied ground truth. To verify LLMs performance at scale, we developed an automated machine-grading pipeline suitable for advanced physics research problems. For example, we handle non-commuting operators that are essential for quantum many-body problems by symbolic manipulation and normal ordering. Our evaluations show that frontier models struggle with all of the problems in the dataset, highlighting a gap in the physical reasoning skills of current LLMs. Notably, experts identified strategies for creating increasingly difficult problems by interacting with the LLMs and exploiting common failure modes. While the highest-performing model, GPT5, correctly solves 30\% of the problems, average performance across 17 models (GPT, Gemini, Claude, DeepSeek, and Llama classes) is only 11.4$\pm$2.1\%. Moreover, our benchmark contains 18 problems that not a single one of the 17 models considered here can correctly solve, and 26 problems that are solved by at most one model. These currently unsolvable problems span the fields of Quantum Monte Carlo, Variational Monte Carlo, and Density Matrix Renormalization Group. Furthermore, we illustrate how incorrect answers sometimes violate fundamental symmetries or have unphysical scaling dimensions. We believe that this benchmark set provides valuable guidance for the future development of language models, aiming to achieve the goal of AI research assistants and tutors.

Applications · Physics

Tai Hoang, Alessandro Trenta, Alessio Gravina, Niklas Freymuth, Philipp Becker, Davide Bacciu, Gerhard Neumann

Learning to simulate complex physical systems from data has emerged as a promising way to overcome the limitations of traditional numerical solvers, which often require prohibitive computational costs for high-fidelity solutions. Recent Graph Neural Simulators (GNSs) accelerate simulations by learning dynamics on graph-structured data, yet often struggle to capture long-range interactions and suffer from error accumulation under autoregressive rollouts. To address these challenges, we propose Information-preserving Graph Neural Simulators (IGNS), a graph-based neural simulator built on the principles of Hamiltonian dynamics. This structure guarantees preservation of information across the graph, while extending to port-Hamiltonian systems allows the model to capture a broader class of dynamics, including non-conservative effects. IGNS further incorporates a warmup phase to initialize global context, geometric encoding to handle irregular meshes, and a multi-step training objective that facilitates PDE matching, where the trajectory produced by integrating the port-Hamiltonian core aligns with the ground-truth trajectory, thereby reducing rollout error. To evaluate these properties systematically, we introduce new benchmarks that target long-range dependencies and challenging external forcing scenarios. Across all tasks, IGNS consistently outperforms state-of-the-art GNSs, achieving higher accuracy and stability under challenging and complex dynamical systems. Our project page: https://thobotics.github.io/neural_pde_matching.

Applications · Robotics

Zeyi Liu, Shuang Li, Eric Cousineau, Siyuan Feng, Benjamin Burchfiel, Shuran Song

Understanding and predicting dynamics of the physical world can enhance a robot's ability to plan and interact effectively in complex environments. While recent video generation models have shown strong potential in modeling dynamic scenes, generating videos that are both temporally coherent and geometrically consistent across camera views remains a significant challenge. To address this, we propose a 4D video generation model that enforces multi-view 3D consistency of generated videos by supervising the model with cross-view pointmap alignment during training. Through this geometric supervision, the model learns a shared 3D scene representation, enabling it to generate spatio-temporally aligned future video sequences from novel viewpoints given a single RGB-D image per view, and without relying on camera poses as input. Compared to existing baselines, our method produces more visually stable and spatially aligned predictions across multiple simulated and real-world robotic datasets. We further show that the predicted 4D videos can be used to recover robot end-effector trajectories using an off-the-shelf 6DoF pose tracker, yielding robot manipulation policies that generalize well to novel camera viewpoints.

Xiaodi Yuan, Fanbo Xiang, Yin Yang 0002, Hao Su 0001

In physics-based simulation of rigid or nearly rigid objects, collisions often become the primary performance bottleneck, particularly when enforcing intersection-free constraints. Previous simulation frameworks rely on primitive-level CCD algorithms. Due to the large number of colliding surface primitives to process, those methods are computationally intensive and heavily dependent on advanced parallel computing resources such as GPUs, which are often inaccessible due to competing tasks or capped threading capacity in applications like policy training for robotics. To address these limitations, we propose a sequential CCD algorithm for convex shapes undergoing constant affine motion. This approach uses the conservative advancement method to iteratively refine a lower-bound estimate of the TOI, exploiting the linearity of affine motion and the efficiency of convex shape distance computation. Our CCD algorithm integrates seamlessly into the ABD framework, achieving a 10-fold speed-up over primitive-level CCD. Its high single-threaded efficiency further enables significant throughput improvements via scene-level parallelism, making it well-suited for resource-constrained environments.

Zizhou Huang, Max Paik, Zachary Ferguson, Daniele Panozzo, Denis Zorin

Barrier potentials gained popularity as a means for robust contact handling in physical modeling and for modeling self-avoiding shapes. The key to the success of these approaches is adherence to geometric constraints, i.e., avoiding intersections, which are the cause of most robustness problems in complex deformation simulation with contact. However, existing barrier-potential methods may lead to spurious forces and imperfect satisfaction of the geometric constraints. They may have strong resolution dependence, requiring careful adaptation of the potential parameters to the object discretizations. We present a systematic derivation of a continuum potential defined for smooth and piecewise smooth surfaces, starting from identifying a set of natural requirements for contact potentials, including the barrier property, locality, differentiable dependence on shape, and absence of forces in rest configurations. Our potential is formulated independently of surface discretization and addresses the shortcomings of existing potential-based methods while retaining their advantages. We present a discretization of our potential that is a drop-in replacement for the potential used in the incremental potential contact formulation [Li et al. 2020], and compare its behavior to other potential formulations, demonstrating that it has the expected behavior. The presented formulation connects existing barrier approaches, as all recent existing methods can be viewed as a variation of the presented potential, and lays a foundation for developing alternative (e.g., higher-order) versions.

Hossein Baktash, Nicholas Sharp, Qingnan Zhou, Alec Jacobson, Keenan Crane

This paper explores the analysis and design of the resting configurations of a rigid body, without the use of physical simulation. In particular, given a rigid body in R3, we identify all possible stationary points, as well as the probability that the body will stop at these points, assuming a random initial orientation and negligible momentum. The forward version of our method can hence be used to automatically orient models, to provide feedback about object stability during the design process, and to furnish plausible distributions of shape orientation for natural scene modeling. Moreover, a differentiable inverse version of our method lets us design shapes with target resting behavior, such as dice with target, nonuniform probabilities. Here we find solutions that would be nearly impossible to find using classical techniques, such as dice with additional unstable faces that provide more natural overall geometry. From a technical point of view, our key observation is that rolling equilibria can be extracted from the Morse-Smale complex of the support function over the Gauss map. Our method is hence purely geometric, and does not make use of random sampling, or numerical time integration. Yet surprisingly, this purely geometric model makes extremely accurate predictions of rest behavior, which we validate both numerically, and via physical experiments. Moreover, for computing rest statistics, it is orders of magnitude faster than state of the art rigid body simulation, opening the door to inverse design—rather than just forward analysis.

Hao Xu 0018, Yinqiao Wang, Niloy J. Mitra, Shuaicheng Liu, Pheng-Ann Heng, Chi-Wing Fu

Hand shadow art is a captivating art form, creatively using hand shadows to reproduce expressive shapes on the wall. In this work, we study an inverse problem: given a target shape, find the poses of left and right hands that together best produce a shadow resembling the input. This problem is nontrivial, since the design space of 3D hand poses is huge while being restrictive due to anatomical constraints. Also, we need to attend to the input's shape and crucial features, though the input is colorless and textureless. To meet these challenges, we design Hand-Shadow Poser, a three-stage pipeline, to decouple the anatomical constraints (by hand) and semantic constraints (by shadow shape): (i) a generative hand assignment module to explore diverse but reasonable left/right-hand shape hypotheses; (ii) a generalized hand-shadow alignment module to infer coarse hand poses with a similarity-driven strategy for selecting hypotheses; and (iii) a shadow-feature-aware refinement module to optimize the hand poses for physical plausibility and shadow feature preservation. Further, we design our pipeline to be trainable on generic public hand data, thus avoiding the need for any specialized training dataset. For method validation, we build a benchmark of 210 diverse shadow shapes of varying complexity and a comprehensive set of metrics, including a novel DINOv2-based evaluation metric. Through extensive comparisons with multiple baselines and user studies, our approach is demonstrated to effectively generate bimanual hand poses for a large variety of hand shapes for over 85% of the benchmark cases.

Michael Liu, Xinlei Wang, Minchen Li

The Material Point Method (MPM) has become a cornerstone of physics-based simulation, widely used in geomechanics and computer graphics for modeling phenomena such as granular flows, viscoelasticity, fracture mechanics, etc. Despite its versatility, the original MPM suffers from cell-crossing instabilities caused by discontinuities in particle-grid transfer kernels. Existing solutions mostly mitigate these issues by adopting smoother shape functions, but at the cost of increased numerical diffusion and computational overhead due to larger kernel support. In this paper, we propose a novel C2-continuous compact kernel for MPM that achieves a unique balance in terms of stability, accuracy, and computational efficiency. Our method integrates seamlessly with Affine Particle-In-Cell (APIC) and Moving Least Squares (MLS) MPM, while only doubling the number of grid nodes associated with each particle compared to linear kernels. At its core is an innovative dual-grid framework, which associates particles with grid nodes exclusively within the cells they occupy on two staggered grids, ensuring consistent and stable force computations. We demonstrate that our method can be conveniently implemented using a domain-specific language, Taichi, or based on open-source GPU MPM frameworks, achieving faster runtime and less numerical diffusion compared to quadratic B-spline MPM. Comprehensive validation through unit tests, comparative studies, and stress tests demonstrates the efficacy of our approach in conserving both linear and angular momentum, handling stiff materials, and scaling efficiently for large-scale simulations. Our results highlight the transformative potential of compact, high-order kernels in advancing MPM's capabilities for stable, accurate, and high-performance simulations.

Zhanyu Yang, Aryamaan Jain, Guillaume Cordonnier, Marie-Paule Cani, Zhaopeng Wang, Bedrich Benes

We introduce Arenite, a novel physics-based approach for modeling sandstone structures. The key insight of our work is that simulating a combination of stress and multi-factor erosion enables the generation of a wide variety of sandstone structures observed in nature. We isolate the key shape-forming phenomena: multi-physics fabric interlocking, wind and fluvial erosion, and particle-based deposition processes. Complex 3D structures such as arches, alcoves, hoodoos, or buttes can be achieved by creating simple 3D structures with user-painted erodable areas and vegetation and running the simulation. We demonstrate the algorithm on a wide variety of structures, and our GPU-based implementation achieves the simulation in less than 5 minutes on a desktop computer for our most complex example.

Filipe de Carvalho Nascimento, Fabricio S. Sousa, Afonso Paiva 0001

Powder-snow avalanches are natural phenomena that result from an instability in the snow cover on a mountain relief. It begins with a dense avalanche core moving fast down the mountain. During its evolution, the snow particles in the avalanche front mix with the air, forming a suspended turbulent cloud of snow dust surrounding the dense snow avalanche. This paper introduces a physically-based framework using the Finite Volume Method to simulate powder-snow avalanches under complex terrains. Specifically, the primary goal is to simulate the turbulent snow cloud dynamics within the avalanche in a visually realistic manner. Our approach relies on a multi-layer model that splits the avalanche into two main layers: dense and powder-snow. The dense-snow layer flow is simulated by solving a type of Shallow Water Equations suited for intricate basal surfaces, known as the Savage-Hutter model. The powder-snow layer flow is modeled as a two-phase mixture of miscible fluids and simulated using Navier-Stokes equations. Moreover, we propose a novel model for the transition layer, which is responsible for coupling the avalanche main layers, including the snow mass injected into the powder-snow cloud from the snow entrainment processes and its injection velocity. In brief, our framework comprehensively simulates powder-snow avalanches, allowing us to render convincing animations of one of the most complex gravity-driven flows.

Mengyun Liu, Kai Bai, Xiaopei Liu

Turbulent boundary layer represents one of the most complex but interesting phenomena in fluid flows. While the generation and alteration of sheared vortices in various interacting scales near the boundary are visually appealing, it is difficult to correctly replicate such phenomena by simulation, especially at high Reynolds numbers. Practical methodologies typically incorporate empirical wall modeling to substantially curtail the computational expenses while retaining physical consistency. Nevertheless, these are predominantly applicable to steady-state flow solvers. While complex scenarios involving dynamic fluid-solid interaction and its application to create time-dependent flow phenomena invariably necessitate unsteady flow solvers, the underlying wall modeling techniques are imprecise, leading to a different formation of near-wall vortices, especially for the highly efficient lattice Boltzmann solver operating on Cartesian grids. In this paper, we propose a novel hybrid near-wall model for the lattice Boltzmann solver, which can handle turbulent boundary layer flows in a simple and efficient manner, inspired by measuring the degree of boundary layer separation. Our model comprises both macroscopic and mesoscopic algebraic models, which collaborate to let the low dissipation lattice Boltzmann solver naturally form the turbulent boundary layer appropriately. By leveraging the multi-resolution technique, accurate simulation outcomes can be obtained. Our model is parameterized to approximate different physical attributes of the solid surface that can potentially influence the boundary layer distribution, and comparable boundary layer flow behaviors can be simulated at various grid resolutions. Rigorous benchmark tests are carried out to validate our model at different grid resolutions by comparing with experimental data and visualizations. We showcase the applications of our new model in both facilitating computational design and generating visual animations, accompanied by specific examples and comparisons with actual experimental setups and photographic images. All demonstrations affirm the physical consistency of our solver even when simulated with a relatively coarse grid resolution.

Wonjong Jang, Yucheol Jung, Gyeongmin Lee, Seungyong Lee 0001

Self-intersection repair in static 3D surface meshes presents unique challenges due to the absence of temporal motion and penetration depth information—two critical elements typically leveraged in physics-based approaches. We introduce a novel framework that transforms local contact handling into a global repair strategy through a combination of local signed tangent-point energies and their gradient diffusion. At the heart of our method is a key insight: rather than computing expensive global repulsive potentials, we can effectively approximate long-range interactions by diffusing energy gradients from local contacts throughout the mesh surface. In turn, resolving complex self-intersections reduces to simply propagating local repulsive energies through standard diffusion mechanics and iteratively solving tractable local optimizations. We further accelerate convergence through our momentum-based optimizer, which adaptively regulates momentum based on gradient statistics to prevent overshooting while maintaining rapid intersection repair. The resulting algorithm handles a variety of challenging scenarios, from shallow contacts to deep penetrations, while providing computational efficiency suitable for interactive applications.

Xinyu Yi, Shaohua Pan 0002, Feng Xu 0005

By learning human motion priors, motion capture can be achieved by 6 inertial measurement units (IMUs) in recent years with the development of deep learning techniques, even though the sensor inputs are sparse and noisy. However, human global motions are still challenging to be reconstructed by IMUs. This paper aims to solve this problem by involving physics. It proposes a physical optimization scheme based on multiple contacts to enable physically plausible translation estimation in the full 3D space where the z-directional motion is usually challenging for previous works. It also considers gravity in local pose estimation which well constrains human global orientations and refines local pose estimation in a joint estimation manner. Experiments demonstrate that our method achieves more accurate motion capture for both local poses and global motions. Furthermore, by deeply integrating physics, we can also estimate 3D contact, contact forces, joint torques, and interacting proxy surfaces. Code is available at https://xinyu-yi.github.io/GlobalPose/.

Mengqi (Mandy) Xia, Zhaoyang Zhang, Sumit Chaturvedi, Yutong Yi, Rundong Wu, Holly E. Rushmeier, Julie Dorsey

The fashion industry has a real need to preview fabric designs using the actual threads they intend to use, ensuring that the designs they envisage can be physically realized. Unfortunately, today's fabric rendering relies on either hand-tuned parameters or parameters acquired from already fabricated cloth. Furthermore, existing curve-based scattering models are not suitable for this problem: they are either not naturally differentiable due to discrete fiber count parameters, or require a more detailed geometry representation, introducing extra complexity. In this work, we bridge this gap by presenting a novel pipeline that captures and digitizes physical threads and predicts the appearance of the fabric based on the weaving pattern. We develop a practical thread scattering model based on simulations of multiple fiber scattering within a thread. Using a cost-efficient multi-view setup, we capture threads of diverse colors and materials. We apply differentiable rendering to digitize threads, demonstrating that our model significantly improves the reconstruction accuracy compared to existing models, matching both reflection and transmission. We leverage a two-scale rendering technique to efficiently render woven cloth. We validate that our digital threads, combined with simulated woven yarn geometry, can accurately predict the fabric appearance by comparing to real samples. We show how our work can aid designs using diverse thread profiles, woven patterns, and textured design patterns.

Xiaoyu Huang, Takara Truong, Yunbo Zhang, Fangzhou Yu, Jean-Pierre Sleiman, Jessica K. Hodgins, Koushil Sreenath, Farbod Farshidian

We present Diffuse-CLoC, a guided diffusion framework for physics-based look-ahead control that enables intuitive, steerable, and physically realistic motion generation. While existing kinematics motion generation with diffusion models offer intuitive steering capabilities with inference-time conditioning, they often fail to produce physically viable motions. In contrast, recent diffusion-based control policies have shown promise in generating physically realizable motion sequences, but the lack of kinematics prediction limits their steerability. Diffuse-CLoC addresses these challenges through a key insight: modeling the joint distribution of states and actions within a single diffusion model makes action generation steerable by conditioning it on the predicted states. This approach allows us to leverage established conditioning techniques from kinematic motion generation while producing physically realistic motions. As a result, we achieve planning capabilities without the need for a high-level planner. Our method handles a diverse set of unseen long-horizon downstream tasks through a single pre-trained model, including static and dynamic obstacle avoidance, motion in-betweening, and task-space control. Experimental results show that our method significantly outperforms the traditional hierarchical framework of high-level motion diffusion and low-level tracking.