论文检索

Applications · Time Series

Zexin Yuan, Qinliang Su, Junxi Xiao

Irregular time series arise ubiquitously in real-world systems, where observations are sparse, asynchronous, and governed by underlying continuous-time dynamics. Existing continuous–discrete state-space models typically rely on path-based variational inference, which is computationally expensive or constrained by restrictive posterior assumptions. We propose SDEVI, a novel framework that performs variational inference directly on the joint distribution over discrete-time observations, while guaranteeing consistency with an underlying continuous process governed by a Stochastic Differential Equation(SDE). SDEVI employs a variational posterior induced by linear time-varying SDEs as a scalable inference backbone. To enable intricate dynamics modeling for real-world data, we introduce non-linear-SDE-induced variational inference and generalize our framework to the complex domain. Extensive experiments across healthcare, physics, climate, and IoT benchmarks demonstrate state-of-the-art performance on interpolation, extrapolation, regression, and classification tasks.

Applications · Computer Vision

Jiachen Tao, Junyi Wu, Haoxuan Wang, Zongxin Yang, Dawen Cai, Yan Yan

We present ReflFlow, a novel framework for high-fidelity rendering of dynamic specular scenes by addressing two key challenges: precise reflection direction estimation and physically accurate modeling. To achieve this, we propose a Residual Material-Augmented 2D Gaussian Splatting representation that models dynamic geometry and material properties, allowing accurate reflection ray computation. Furthermore, we introduce a Dynamic Environment Gaussian and a hybrid rendering pipeline that decomposes rendering into diffuse and specular components, enabling physically informed specular synthesis via rasterization and ray tracing. Finally, we devise a coarse-to-fine training strategy to improve optimization stability and promote physically meaningful decomposition. Extensive experiments on dynamic scene benchmarks demonstrate that ReflFlow outperforms prior methods quantitatively and qualitatively, producing sharper and more realistic specular reflections in complex dynamic environments.

Applications · Health / Medicine

Ruihan Guo, Chaoran Cheng, Zhanghan Ni, Neil He, Bangji Yang, Ge Liu

Predicting protein mutation effects is fundamental to protein engineering and disease variant interpretation, yet experimental mutation data remain accurate but extremely sparse. Large-scale computational augmentation offers scalability, but introduces heterogeneous and poorly calibrated supervision signals arising from distinct modeling paradigms. We construct a PDB-wide, structure-aligned mutation augmentation dataset that exhaustively enumerates single-site substitutions and aligns mutation signals from physics-based energy models, protein language models, and structure-conditioned inverse folding models. Large-scale analysis under a unified mutation preference representation reveals substantial differences in preference structure, confidence, and cross-model agreement, indicating that disagreement is pervasive and reflects conflicting inductive biases rather than random noise. Motivated by these observations, we propose an unsupervised multi-source mutation preference distillation framework that learns from relative mutation preferences while explicitly modeling cross-source disagreement. Without using any experimental mutation labels during training, our approach achieves consistent improvements on the ProteinGym benchmark over zero-shot baselines and naive multi-source fusion strategies. We release the dataset and evaluation pipeline to support reproducible studies of protein mutation effects.

Yanjiang Guo, Tony Lee, Lucy Xiaoyang Shi, Jianyu Chen, Percy Liang, Chelsea Finn

The goal of this paper is to improve the performance and reliability of vision-language-action (VLA) models through iterative online interaction. Since collecting policy rollouts in the real world is expensive, we investigate whether a learned simulator—specifically, an action-conditioned video generation model—can be used to generate additional rollout data. Unfortunately, existing world models lack the physical fidelity necessary for policy improvement: they are predominantly trained on demonstration datasets that lack coverage of many different physical interactions (particularly failure cases) and struggle to accurately model small yet critical physical details in contact-rich object manipulation. We propose a simple iterative improvement algorithm that uses real-world roll-out data to improve the fidelity of the world model, which can then, in turn, be used to generate supplemental synthetic data for improving the VLA model. In our experiments on a real robot, we use this approach to improve the performance of a state-of-the-art VLA model on multiple downstream tasks. We achieve a 39.2\% absolute success rate improvement over the base policy and 11.6\% improvement from training with the generated synthetic rollouts. Videos can be found at this anonymous website: \url{https://sites.google.com/view/vla-w}.

Andrew Gordon, Daniel Murfet, William Snell, Garrett Baker, George Wang, Stan van Wingerden

Spectroscopy infers the internal structure of physical systems by measuring their response to perturbations. We apply this principle to neural networks: perturbing the data distribution by upweighting a token $y$ in context $x$, we measure the model's response via susceptibilities $\chi_{xy}$, which are covariances between component-level observables and the perturbation computed over a localized Gibbs posterior via stochastic gradient Langevin dynamics (SGLD). Theoretically, we show that susceptibilities decompose as a sum over \emph{modes} of the data distribution, explaining why tokens that follow their contexts ``for similar reasons'' cluster together in susceptibility space. Empirically, we apply this methodology to Pythia-14M, developing a conductance-based clustering algorithm that identifies 510 interpretable clusters ranging from grammatical patterns to code structure to mathematical notation. Comparing to sparse autoencoders, 50\% of our clusters match SAE features, validating that both methods recover similar structure.

Probabilistic Methods · Structure Learning

Shida Liu, Abhishek Gupta, Sumit Sinha, L Mahadevan

Symmetry is central to modern machine learning and physics: invariances and equivariances improve sample efficiency, robustness, and out-of-distribution generalization, while symmetry principles guide scientific modeling. Yet for stochastic dynamical systems, the relevant continuous symmetries are rarely known, and symmetry discovery for SDEs has remained essentially unexplored. We introduce *LieStoNet*, an end-to-end, *prior-free* framework for discovering Lie-point symmetries of SDEs directly from spatiotemporal trajectories, without prespecifying symmetry groups, templates, or canonical coordinates. Building on the seminal SDE Lie-symmetry theory of Gaeta and Quintero (1999), which formalizes Lie-point SDE symmetries and their relation to Fokker-Planck symmetries, LieStoNet learns neural surrogates for drift and diffusion from increments, then learns projectable generators by enforcing the SDE determining equations, separately regularizing for closure under Lie brackets, adherence to the Lie algebra axioms (bilinearity, antisymmetry, Jacobi), and a non-redundant independent basis. The surrogate also defines an associated Fokker-Planck equation, enabling optional discovery of its Lie-point symmetries in parallel. Across multiple canonical SDEs with known analytic symmetries, LieStoNet recovers generators consistent with the ground-truth symmetry algebra, providing interpretable symmetry discovery for noisy dynamics.

General Machine Learning · Hardware and Software

Yuxuan Yang, Feiyang Ren, Bowen Zeng, Dalin Zhang, Jinpeng Chen, Gang Chen, Huan Li

Long-context LLM inference faces a fundamental conflict: head-adaptive compression algorithms (e.g., Top-$p$ nucleus sampling) offer superior accuracy by dynamically fluctuating memory budgets, yet modern inference engines (e.g., vLLM) demand rigid, static memory patterns to leverage CUDA Graphs and PagedAttention. We resolve this ``Static-Dynamic'' mismatch with HARD-KV, a unified framework that that bridges dynamic selection with rigid system constraints. HARD-KV introduces a Cascade Cache hierarchy, managing the token lifecycle across dense, sparse, and condensed tiers. Crucially, we propose a Logits Calibration mechanism that normalizes diverse importance metrics into a unified probability space, enabling consistent Top-$p$ budgeting across heterogeneous heads. To bridge the efficiency gap, we offer a system-level solution, which rewrites fragmented, dynamic indices into contiguous physical layouts compatible with high-performance inference engine. Extensive experiments on math-reasoning benchmarks (AIME, U-Math) verify that HARD-KV achieves up to 2$\times$ throughput improvement over static baselines while maintaining high-fidelity generation in 10k+ token scenarios. Our code will be made publicly available.

Applications · Chemistry, Physics, and Earth Sciences

Tianchi Yu, Ivan Oseledets

For low-dimensional problems ($d\leq3$), spectral methods can achieve exceptionally high accuracy. For middle-dimensional problems ($4 \leq d \lesssim 10$), spectral methods remain feasible through specific techniques such as sparse grids or hyperbolic cross. However, for high-dimensional problems ($d\gg 10$), spectral methods suffer from the curse of dimensionality. Physics-informed neural networks (PINNs) have emerged as a promising approach to overcome this challenge, offering scalability to high dimensions, but often suffer from limited accuracy and efficiency. Recently proposed spectral-informed neural networks (SINNs) combine spectral methods with PINNs, operating directly in the spectral domain to avoid spatial derivative computations and to reduce memory consumption. In this work, we introduce Modified SINNs, which integrate coefficient decay scaling and basis embeddings motivated by harmonic analysis to enhance accuracy in high-dimensional problems and enable accurate approximation of unknown spectral coefficients. Numerical experiments on steady and time-dependent partial differential equations demonstrate that Modified SINNs outperform sparse grid spectral methods on middle-dimensional problems with incomplete spectral information and achieve superior accuracy compared to PINNs on high-dimensional problems.

Applications · Chemistry, Physics, and Earth Sciences

Vansh Ramani, Har Ashish Arora, Dhairya Kuchhal, Sayan Ranu, Tarak Karmakar

High-fidelity solubility prediction is fundamental to pharmaceutical development and environmental partitioning, where accurate modeling must couple molecular structure with thermodynamic behavior across diverse chemical environments. However, recent advancements have been dominated by deep learning architectures that often sacrifice physical interpretability for predictive power. We challenge this trend by showing that state-of-the-art performance does not require such non-transparent architectures. To address this, we introduce $Dissolvr$, a transparent framework for molecular solubility prediction. In addition, we perform a comprehensive literature review and a benchmarking study against various methods. We show that $Dissolvr$ approaches the aleatoric limit of experimental uncertainty and achieves OOD generalization through structural invariance derived by mapping molecules to physically-grounded descriptors. Then, we present an LLM-assisted post-hoc explanation pipeline that bridges the gap between symbolic model artifacts and chemically grounded narratives. Finally, a comparative benchmark of a survey involving 22 expert chemists reveals that expert evaluators provide deep insights.

Probabilistic Methods · Bayesian Models and Methods

Yujia Guo, Daolang Huang, Xinyu Zhang, Sammie Katt, Samuel Kaski, Ayush Bharti

Bayesian experimental design (BED) is a principled framework for data-efficient design of sequential experiments. However, existing BED methods are unable to adapt to dynamic constraints inherent in real-world tasks due to budget limitations, varying costs, or physical constraints that restrict how designs evolve over time. In this paper, we introduce a novel approach to BED that enables constrained optimization of experimental designs by combining offline pre-training of an amortized policy and a posterior network with online multi-step lookahead planning using scenario trees. We empirically demonstrate that our method yields substantially more informative design sequences than existing methods across a range of constrained BED tasks, while incurring only a modest additional computational overhead.

Applications · Time Series

Fengming Zhang, Wenjie Du, Huan Zhang, Ke Yu, Shen Qu

Time series imputation benefits from leveraging cross-feature correlations, yet existing attention based methods re-discover feature relationships at each layer, lacking persistent anchors to maintain consistent representations. To address this, we propose HELIX, which assigns each feature a learnable feature identity, a persistent embedding that captures intrinsic semantic properties throughout the network. Unlike graph-based methods that rely on predefined topology and assume homogeneous spatial relationships, HELIX learns arbitrary feature dependencies end-to-end from temporal co-variation, naturally handling datasets where features mix spatial locations with semantic variables. Integrated with hybrid temporal-feature attention, HELIX achieves the state-of-the-art performance, ranking first among 17 methods across 21 experimental settings. Furthermore, our mechanistic analysis reveals that feature attention progressively aligns with underlying physical structure across layers, demonstrating more effectively exploits cross-feature dependencies for imputation.

Applications · Chemistry, Physics, and Earth Sciences

Yingheng Wang, Tao Yu, Shufeng Kong, Yingheng Wang, John Gregoire, Carla Gomes

Structured scientific spectra encode rich physical information while satisfying hard constraints such as conservation and spectral geometry. We study a canonical example, the electronic density of states (eDOS), whose accurate prediction is central to materials discovery. Prior methods often (i) decouple band gaps from eDOS, (ii) violate total-state conservation, or (iii) collapse crystals into global embeddings that obscure atom-projected contributions. We introduce \textbf{DeepSciReasoner}, a design paradigm for deep scientific reasoning under physical constraints. Instantiated for eDOS prediction, DeepSciReasoner combines structure-aware spectrum decoding with constraint-preserving physical reasoning, in this case, mass-conserving iterative refinement. It substantially improves eDOS accuracy while maintaining physical validity, enabling reliable high-throughput screening. Beyond eDOS, DeepSciReasoner offers a reusable blueprint for predicting structured scientific spectra under hard physical constraints.

Applications · Chemistry, Physics, and Earth Sciences

Andy Xu, Rohan Desai, Larry Wang, Ethan Ritz, Gabriel Hope

Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising approach to improve correctness in LLMs, however, in many scientific problems, the objective is not necessarily to produce \textit{the} correct answer, but instead to produce a diverse array of candidates which satisfy a set of constraints. We study this challenge in the context of materials generation. To this end, we introduce PLaID++, an LLM post-trained for stable and property-guided crystal generation. We find that applying naive preference optimization to a coordinate-based crystal representation leads to mode collapse. Hence, we introduce a compact, symmetry-informed Wyckoff text representation which improves computational efficiency and encourages generalization from physical priors. By encoding symmetry constraints directly into text and guiding model outputs towards desirable chemical space, PLaID++ generates structures that are thermodynamically stable, unique, and novel at a $>$50\% greater rate than prior methods. We further demonstrate that unified training across conditional and unconditional tasks are mutually beneficial in data-sparse regimes. Our work demonstrates the potential of adapting post-training techniques from natural language processing to materials design, paving the way for targeted and efficient discovery of novel materials.

Deep Learning · Foundation Models

Fang Wu, Li Erran Li, Weihao Xuan, Heli Qi, Zeqi Zhou, Hanqun CAO, Heng-Jui Chang, Haokai Zhao, Jian Ma, Zijian Ma 等

Recent advances in generative diffusion and flow-matching models have revolutionized molecular design, enabling the creation of novel proteins, small molecules, and RNA sequences with unprecedented fidelity. Yet, these models remain intuitive rather than intelligent—they generate without reasoning. \textbf{ThinkProteo} reimagines generative science by introducing reasoning-guided diffusion models that think step-by-step, akin to how a scientist hypothesizes, tests, and refines molecular ideas. By embedding chain-of-thought (CoT) reasoning into the continuous generative trajectory, ThinkProteo transforms diffusion into a process of thought: each denoising step becomes an interpretable act of molecular reasoning guided by structural, energetic, and functional objectives. This framework bridges symbolic reasoning and physical generation, yielding models that not only design molecules but also explain why they work. We envision ThinkProteo as a foundation for cognitive generative chemistry—uniting the creativity of diffusion models with the deliberation of human reasoning to accelerate the discovery of safe and effective therapeutics.

Applications · Chemistry, Physics, and Earth Sciences

Yanming Xia, Angelica Aviles-Rivero

Neural operators perform well on structured domains, yet their behaviour on irregular geometries remains poorly understood. We show that this limitation is not merely an encoding issue, but a depth-wise failure mode inherent to deep operator architectures. We formalise the *Geometric Forgetting Hypothesis*: due to the Markovian structure of operator layers and their reliance on global mixing mechanisms, neural operators progressively lose access to domain geometry as depth increases. Using layer-wise geometric probing, we demonstrate that both spectral and attention-based operators systematically lose geometric fidelity. We show that this geometric forgetting degrades accuracy, stability, and generalisation. To counteract it, we introduce a lightweight *geometry memory injection* mechanism that restores geometric constraints at intermediate depths with minimal architectural overhead. This simple intervention consistently mitigates forgetting and exposes a *geometric shortcut* instability in transformer-based operators, revealing that geometric retention is a structural requirement rather than a design choice.

Deep Learning · Sequential Models, Time series

Tomoyoshi Kimura, Denizhan Kara, Jinyang Li, Hongjue Zhao, Yigong Hu, Yizhuo Chen, Xiaomin OUYANG, Shengzhong Liu, Tarek Abdelzaher

Internet of Things (IoT) systems continuously collect heterogeneous sensing signals from ubiquitous sensors to support intelligent applications such as human activity analysis, emotion monitoring, and environmental perception. These signals are inherently non-stationary and multi-scale, posing unique challenges for standard tokenization techniques. This paper proposes Dywave, a dynamic tokenization framework for IoT sensing signals that constructs compact input representations aligned with intrinsic temporal structures and underlying physical events. Dywave leverages wavelet-based hierarchical decomposition, identifies meaningful temporal boundaries corresponding to underlying semantic events, and adaptively compresses redundant intervals while preserving temporal coherence. Extensive evaluations on five real-world IoT sensing datasets across activity recognition, stress assessment, and nearby object detection demonstrate that Dywave outperforms state-of-the-art methods by up to 12\% in accuracy, while improving computational efficiency by reducing input token lengths by up to 75\% across mainstream sequence models. Moreover, Dywave exhibits improved robustness to domain shifts and varying sequence lengths.

Applications · Chemistry, Physics, and Earth Sciences

Tianmeng Hu, Biao Luo, Ke Li

Designing RNA sequences that reliably fold into specific secondary structures is essential for understanding their biological functions but remains a challenging computational problem. We propose CocoRNA, a cooperative multi-agent reinforcement learning framework for RNA inverse design. CocoRNA simplifies the design task by decomposing it into smaller sub-problems, each solved collaboratively by multiple agents. This approach reduces the complexity of the problem and improves the exploration of design policies. During training, a centralized critic uses global structural information to guide the agents, enabling them to jointly optimize their design strategies. As a result, CocoRNA learns high-quality RNA design policies that generalize effectively to unseen structures without additional training. Experiments on the Rfam dataset demonstrate that CocoRNA substantially outperforms state-of-the-art methods in both success rate and design speed. Further experiments on other biological sequence design tasks highlight the effectiveness and broad potential of CocoRNA for complex design tasks.

Applications · Chemistry, Physics, and Earth Sciences

Zherui Yang, Haiyang Xin, Tao Du, Ligang Liu

Neural operators have emerged as data-driven surrogates for solving partial differential equations (PDEs), and their success hinges on efficiently modeling the long-range, global coupling among spatial points induced by the underlying physics. In many PDE regimes, the induced global interaction kernels are empirically compressible, exhibiting rapid spectral decay that admits low-rank approximations. We leverage this observation to unify representative global mixing modules in neural operators under a shared low-rank template: compressing high-dimensional pointwise features into a compact latent space, processing global interactions within it, and reconstructing the global context back to spatial points. Guided by this view, we introduce Low-Rank Spatial Attention (LRSA) as a clean and direct instantiation of this template. Crucially, unlike prior approaches that often rely on non-standard aggregation or normalization modules, LRSA is built purely from standard Transformer primitives, i.e., attention, normalization, and feed-forward networks, yielding a concise block that is straightforward to implement and directly compatible with hardware-optimized kernels. In our experiments, such a simple construction is sufficient to achieve high accuracy, yielding an average error reduction of over 17\% relative to second-best methods, while remaining stable and efficient in mixed-precision training.

Applications · Chemistry, Physics, and Earth Sciences

Alejandro Queiruga, Theo Gutman-Solo, Shuai Jiang

While there are many applications of machine learning (ML) to scientific problems that \emph{look} promising, the eye test can be misleading compared to the quantitative values. Using numerical analysis techniques, we rigorously quantify the accuracy, convergence rates, and generalization bounds of certain ML models applied to linear differential equations (DEs) for parameter discovery or solution finding. Beyond the quantity and discretization of data, we identify that the {function space} of the data is critical to the generalization of the model which can lead to divergence. Similar lack of generalization is empirically demonstrated for commonly used models. Surprisingly, we find that different classes of models can exhibit opposing generalization behaviors. Based on our theoretical analysis, we also introduce a new mechanistic interpretability lens on scientific models whereby Green's function representations can be extracted from the weights of black-box models. Our results inform a new cross-validation technique for measuring generalization in physical systems, and can be useful as a benchmark of future methods.

Social Aspects · Safety

Chuancheng Shi, shangze li, Wenjun Lu, Wenhua Wu, Fei Shen, Cong Wang, Zifeng Cheng, Tat-Seng Chua

Despite their capabilities, large foundation models (LFMs) remain susceptible to adversarial manipulation. Current defenses predominantly rely on the ``locality hypothesis", suppressing isolated neurons or features. However, harmful semantics act as distributed, cross-layer circuits, rendering such localized interventions brittle and detrimental to utility. To bridge this gap, we propose \textbf{TraceRouter}, a path-level framework that traces and disconnects the causal propagation circuits of illicit semantics. TraceRouter operates in three stages: (1) it pinpoints a sensitive onset layer by analyzing attention divergence; (2) it leverages sparse autoencoders (SAEs) and differential activation analysis to disentangle and isolate malicious features; and (3) it maps these features to downstream causal pathways via feature influence scores (FIS) derived from zero-out interventions. By selectively suppressing these causal chains, TraceRouter physically severs the flow of harmful information while leaving orthogonal computation routes intact. Extensive experiments demonstrate that TraceRouter significantly outperforms state-of-the-art baselines, achieving a superior trade-off between adversarial robustness and general utility. Our code will be publicly released. WARNING: This paper contains unsafe model responses.