The RAW domain image super-resolution faces two critical challenges: the physical impossibility of capturing native high-quality RAW references with a resolution-limited camera and the limitations of neural networks, including inefficient residual layer utilization and spectral bias in feature learning. This paper proposes a strategy combining physics-based imaging simulation and neural networks to jointly address these challenges. First, we develop a rapid imaging simulation system based on our proposed subgraph decomposition technology. It generates camera-specific degraded and clean RAW image pairs at multiple resolutions. Second, we design a LatentKAN network, featuring an iterative feature fusion network that extracts additional beneficial information through stage-wise supervision and a multi-layer Kolmogorov Arnold network that suppresses spectral bias via learnable activation functions. Ultimately, our strategy demonstrates significant advantages, achieving an average 0.8 dB PSNR improvement across all SR scales compared to state-of-the-art methods, thereby establishing a new paradigm for camera-specific super-resolution tasks.
论文检索
输入标题、作者或关键词，从 4,294 篇学术成果中精准定位
Table structure recognition (TSR), the task of extracting logical and physical structures from table images, is critical for document understanding. Current end-to-end image-to-text methods typically employ a top-down strategy where physical structure prediction depends on the logical decoder's output sequence. However, this process often suffers from training instability and misalignment between predicted bounding boxes and ground-truth cell positions. To address this issue, we propose G2LFormer, a novel transformer-based framework that employs a ''Global-to-Local'' query enhancement strategy. Specifically, G2LFormer introduces a Vision-guided Query Enhancer to integrate both textual and visual modalities, significantly improving the overall query representation capability and boosting prediction accuracy. Additionally, we design a Multi-scale Manhattan Vision-guider that leverages a spatial attenuation matrix to guide each query towards its corresponding cell location, effectively balancing local and global information for more precise bounding box generation. Extensive experiments on benchmark datasets demonstrate G2LFormer's superior performance, while ablation studies confirming the significant contribution of each proposed module in achieving state-of-the-art results. The source code and model have been released at: https://github.com/Hzbupahaozi/G2LFormer.
Spatiotemporal Degradation-Aware 3D Gaussian Splatting for Realistic Underwater Scene Reconstruction
PDF ↗Reconstructing realistic underwater scenes from underwater video remains a meaningful yet challenging task in the multimedia domain. The inherent spatiotemporal degradations in underwater imaging, including caustics, flickering, attenuation, and backscattering, frequently result in inaccurate geometry and appearance in existing 3D reconstruction methods. While a few recent works have explored underwater degradation-aware reconstruction, they often address either spatial or temporal degradation alone, falling short in more real-world underwater scenarios where both types of degradation occur. We propose MartineSTD-GS, a novel 3D Gaussian Splatting-based framework that explicitly models both temporal and spatial degradations for realistic underwater scene reconstruction. Specifically, we introduce two paired Gaussian primitives: Intrinsic Gaussians represent the true scene, while Degraded Gaussians render the degraded observations. The color of each Degraded Gaussian is physically derived from its paired Intrinsic Gaussian via a Spatiotemporal Degradation Modeling (SDM) module, enabling self-supervised disentanglement of realistic appearance from degraded images. To ensure stable training and accurate geometry, we further propose a Depth-Guided Geometry Loss and a Multi-Stage Optimization strategy. We also construct a simulated benchmark with diverse spatial and temporal degradations and ground-truth appearances for comprehensive evaluation. Experiments on both simulated and real-world datasets show that MarineSTD-GS robustly handles spatiotemporal degradations and outperforms existing methods in novel view synthesis with realistic, water-free scene appearances.
Hate speech on social media threatens the mental and physical well-being of individuals and contributes to real-world violence. Resharing is an important driver behind the spread of hate speech on social media. Yet, little is known about who reshares hate speech and what their characteristics are. In this paper, we analyze the role of user characteristics in hate speech resharing across different types of hate speech (e.g., political hate). For this, we first cluster hate speech posts using large language models into different types of hate speech. Then we model the effects of user attributes on users' probability to reshare hate speech using an explainable machine learning model. To do so, we apply debiasing to control for selection bias in our observational social media data and further control for the latent vulnerability of users to hate speech. We find that, all else equal, users with fewer followers, fewer friends, fewer posts, and older accounts share more hate speech. This shows that users with little social influence tend to share more hate speech. Further, we find substantial heterogeneity across different types of hate speech. For example, racist and misogynistic hate is spread mostly by users with little social influence. In contrast, political anti-Trump and anti-right-wing hate is reshared by users with larger social influence. Overall, understanding the factors that drive users to share hate speech is crucial for detecting individuals at risk of engaging in harmful behavior and for designing effective mitigation strategies. Disclaimer: This work contains terms that are offensive and hateful.
Path-LLM: A Multi-Modal Path Representation Learning by Aligning and Fusing with Large Language Models
PDF ↗The advancement of intelligent transportation systems has led to a growing demand for accurate path representations, which are essential for tasks such as travel time estimation, path ranking, and trajectory analysis. However, traditional path representation learning (PRL) methods often focus solely on single-modal road network data, overlooking important physical and regional factors that influence real-world traffic dynamics. To overcome this limitation, we introduce Path-LLM, a multi-modal path representation learning model that integrates large language models (LLMs) into PRL. Our approach leverages LLMs to interpret both topological and textual data, enabling robust multi-modal path representations. To effectively align and merge these modalities, we propose TPalign, a contrastive learning-based pretraining strategy that ensures alignment within the embedding space. We then present TPfusion, a multimodal fusion module that dynamically adjusts the weight of each modality before integration. To further optimize LLM training, we introduce a Two-stage Overlapping Curriculum Learning (TOCL) approach, which progressively increases the complexity of the training data. Finally, we evaluate Path-LLM on three real-world datasets across traditional PRL downstream tasks, achieving up to a 61.84% improvement in path ranking performance on the Xi'an dataset. Additionally, Path-LLM demonstrates superior performance in both few-shot and zero-shot learning scenarios. Our code is available at: https://github.com/decisionintelligence/Path-LLM.
Centrality measures are essential for identifying important nodes and edges in networks. In this paper, we focus on two forest-based centrality measures on undirected graphs: forest node centrality (FNC) and forest edge centrality (FEC), which capture the influence of nodes and edges through their participation in spanning forests. Both centrality measures can be represented using entries of the forest matrix. To address the challenge of computing the two measures on large networks, we propose two scalable algorithms from different perspectives. The first algorithm IFGN combines two variance reduction techniques to approximate the entries of the forest matrix, applicable to both FNC and FEC.The second algorithm FECE incorporates a new physical interpretation of FEC, allowing for a better overall estimation. We provide error guarantees for both algorithms and demonstrate their efficiency and effectiveness through extensive experiments on various real-world networks.
Browser APIs such as WebHID, WebUSB, Web Serial, and Web MIDI enable web applications to interact directly with external devices. The support of such APIs in Chromium-based browsers, such as Chrome and Edge, radically changes the threat model for peripherals and increases the attack surface. In the past, devices could assume a trusted host, i.e., the operating system. Now, the host is a potentially malicious website and cannot be trusted. We show how this changed threat model leads to security and privacy problems, up to a complete compromise of the operating system. While the API specifications list initial security considerations, they shift the responsibility to (unprepared) device vendors. We systematically analyze the security implications of external devices exposed by such new APIs. By reverse-engineering peripheral devices of several popular widespread vendors, we show that many vendors allow controlling devices via Web APIs up to reprogramming or even fully replacing the firmware. Consequently, web attackers can reprogram devices with malicious payloads and custom firmware without requiring any physical interaction. To demonstrate the security implications, we build several full-chain exploits, leading to arbitrary code execution on the victim system, circumventing the browser sandbox. Our research shows that browser security should not rely on the secure implementation of third-party hardware.
Spatial-Temporal Analysis of Collective Emotional Resonance in China During Global Health Crisis
PDF ↗The 21st century has already witnessed so many outbreaks with pandemic potential, including SARS (2002), H1N1 (2009), MERS (2012), Ebola (2014), Zika virus (2015), and the COVID-19 pandemic (2019). Using 60 million geotagged Sina Weibo tweets covering over 20 million active accounts, we investigate the collective emotional dynamics on social media in the most recent global pandemic, i.e., COVID-19. This research features two highlights: (1) It focuses on the Chinese population located in the initial epicenter of the pandemic. (2) It examines the initial year after the pandemic outbreak, a critical period where emotions were most intense due to the uncertainty and rapid developments related to the crisis. Using cross-disciplinary methods, we reveal a positive connection between online emotional resonance and geographic proximity, demonstrating a direct mapping between virtual network distances and physical spatial embedding. We propose a percolation-based index to measure the nationwide emotional resonance level with which we illustrate the significant economic impact of the global health issue. Finally, we identify a leader-follower pattern in emotional resonance fluctuations based on time-lag emotion correlations, revealing that less active regions play a crucial role in leading and responding to emotional changes. In the face of long COVID and emerging global health crises, our analysis elucidates how collective emotional resonance evolves, providing potential directions for online opinion interventions during global shocks.
Graph ODEs and Beyond: A Comprehensive Survey on Integrating Differential Equations with Graph Neural Networks
PDF ↗Graph Neural Networks (GNNs) and differential equations (DEs) are two rapidly advancing areas of research that have shown remarkable synergy in recent years. GNNs have emerged as powerful tools for learning on graph-structured data, while differential equations provide a principled framework for modeling continuous dynamics across time and space. The intersection of these fields has led to innovative approaches that leverage the strengths of both, enabling applications in physics-informed learning, spatiotemporal modeling, and scientific computing. This survey aims to provide a comprehensive overview of the burgeoning research at the intersection of GNNs and DEs. We will categorize existing methods, discuss their underlying principles, and highlight their applications across domains such as molecular modeling, traffic prediction, and epidemic spreading. Furthermore, we identify open challenges and outline future research directions to advance this interdisciplinary field. A comprehensive paper list is provided at https://github.com/Emory-Melody/Awesome-Graph-NDEs.
In recent years, the rapid advancement of Artificial Intelligence (AI) technologies, particularly Large Language Models (LLMs), has revolutionized the paradigm of scientific discovery, establishing AI-for-Science (AI4Science) as a dynamic and evolving field. However, there is still a lack of an effective framework for the overall assessment of AI4Science, particularly from a holistic perspective on data quality and model capability. Therefore, in this study, we propose SciHorizon, a comprehensive assessment framework designed to benchmark the readiness of AI4Science from both scientific data and LLM perspectives. First, we introduce a generalizable framework for assessing AI-ready scientific data, encompassing four key dimensions-Quality, FAIRness, Explainability, and Compliance-which are subdivided into 15 sub-dimensions. Drawing on data resource papers published between 2018 and 2023 in peer-reviewed journals, we present recommendation lists of AI-ready datasets for Earth, Life, and Materials Sciences, making a novel and original contribution to the field. Concurrently, to assess the capabilities of LLMs across multiple scientific disciplines, we establish 16 assessment dimensions based on five core indicators-Knowledge, Understanding, Reasoning, Multimodality, and Values-spanning Mathematics, Physics, Chemistry, Life Sciences, and Earth and Space Sciences. Using the developed benchmark datasets, we have conducted a comprehensive evaluation of over 50 representative open-source and closed-source LLMs. All the results are publicly available and can be accessed online at www.scihorizon.cn/en.
Scientific computing for large deformation of elastic-plastic solids is critical for numerous real-world applications. Classical numerical solvers rely primarily on local discrete linear approximation and are constrained by an inherent trade-off between accuracy and efficiency. Recently, deep learning models have achieved impressive progress in solving the continuum mechanism. While previous models have explored various architectures and constructed coefficient-solution mappings, they are designed for general instances without considering specific problem properties and hard to accurately handle with complex elastic-plastic solids involving contact, loading and unloading. In this work, we take stretch bending, a popular metal fabrication technique, as our case study and introduce LaDEEP, a deep learning-based surrogate model for La rge De formation of Elastic-Plastic Solids. We encode the partitioned regions of the involved slender solids into a token sequence to maintain their essential order property. To characterize the physical process of the solid deformation, a two-stage Transformer-based module is designed to predict the deformation with the sequence of tokens as input. Empirically, LaDEEP achieves five magnitudes faster speed than finite element methods with a comparable accuracy, and gains 20.47% relative improvement on average compared to other deep learning baselines. We have also deployed our model into a real-world industrial production system, and it has shown remarkable performance in both accuracy and efficiency. Code is available at https://github.com/therontau0054/LaDEEP.
Multi-scale Physics-informed Transformer With Spatio-temporal Feature Adapter For Extreme Precipitation Nowcasting
PDF ↗Extreme precipitation, as a core causative factor of meteorological disasters, poses significant challenges for accurate short-term forecasting due to the chaotic nature of precipitation systems and their multi-scale spatio-temporal evolution. Traditional numerical models are notably affected by error accumulation, while existing deep learning models still face dual limitations in physical fidelity and multi-scale feature extraction. To Address these issues, we propose an innovative Multi-scale Physics-informed Transformer with spatio-temporal feature adapter for extreme precipitation nowcasting, termed MPFormer. Our framework comprises two core components: the deterministic Evolution Network and the stochastic Generative Network. The Evolution Network integrates a novel Scale-Aware Temporal Residual Modulation Transformer (STRMT) encoder that captures multi-scale storm dynamics through residual temporal attention. The Generative Network introduces spatio-temporal adapters as lightweight transfer modules for probabilistic modeling. We develop a Multi-scale Physics-informed Loss with three innovations: 1) dynamic weight scheduling for feature fusion, 2) physical constraints preserving storm evolution patterns, and 3) entropy-based uncertainty calibration. Experiments based on MRMS radar data from North America demonstrate that the model can generate high-resolution forecasts (2km grid) with a 3-hour lead time over an area of 2048×2048 square kilometers. Compared to the state-of-the-art technologies, the proposed framework shows significant effectiveness and superiority in metrics such as CSIN, offering a new paradigm that combines physical interpretability with engineering practicality for extreme weather warnings and disaster prevention in smart cities.
Simulation of fluid flows is crucial for modeling physical phenomena like meteorology, aerodynamics, and biomedicine. Classical numerical solvers often require fine spatiotemporal grids to satisfy stability, consistency, and convergence conditions, leading to substantial computational costs. Although machine learning has demonstrated better efficiency, they typically suffer from issues of interpretability, generalizability, and data dependency. Hence, we propose a learnable and differentiable finite volume solver, called LDSolver, designed for efficient and accurate simulation of fluid flows on spatiotemporal coarse grids. LDSolver comprises two key components: (1) a differentiable finite volume solver, and (2) an learnable module providing equivalent approximation for fluxes (derivatives and interpolations), and temporal error correction on coarse grids. Even with limited training data (e.g., only a few trajectories), our model could accelerate the simulation while maintaining a high accuracy with superior generalizability. Experiments on different flow systems (e.g., Burgers, decaying, forced and shear flows) show that LDSolver achieves state-of-the-art performance, surpassing baseline models with notable margins.
Combining Denoised Neural Network and Genetic Symbolic Regression for Memory Behavior Modeling via Dynamic Asynchronous Optimization
PDF ↗Memory behavior modeling is a key topic in cognitive psychology and education. Traditional approaches use experimental data to build memory equations, but these models often lack precision and are debated in form. Recently, data-driven methods have improved predictive accuracy but struggle with interpretability, limiting cognitive insights. Although knowledge-informed neural networks have succeeded in fields like physics, their use in behavior modeling is still limited. This paper proposes a Self-evolving Psychology-informed Neural Network (SPsyINN), which leverages classical memory equations as knowledge modules to constrain neural network training. To address challenges such as the difficulty in quantifying descriptors and the limited interpretability of classical memory equations, a genetic symbolic regression algorithm is introduced to conduct evolutionary searches for more optimal expressions based on classical memory equations, enabling the mutual progress of the knowledge module and the neural network module. Specifically, the proposed approach combines genetic symbolic regression and neural networks in a parallel training framework, with a dynamic joint optimization loss function ensuring effective knowledge alignment between the two modules. Then, for addressing the training efficiency differences arising from the distinct optimization methods and computational hardware requirements of genetic algorithms and neural networks, an asynchronous interaction mechanism mediated by proxy data is developed to facilitate effective communication between modules and improve optimization efficiency. Finally, a denoising module is integrated into the neural network to enhance robustness against data noise and improve generalization performance. Experimental results on five large-scale real-world memory behavior demonstrate that SPsyINN outperforms state-of-the-art methods in predictive accuracy. Ablation studies confirm the model's co-evolution capability, improving accuracy while discovering more interpretable memory equations, showing its potential for psychological research. Our code is released at: https://github.com/JiaqiDijon/SPsyINN
Predicting changes in binding free energy (ΔΔ G) is a vital task in protein engineering and protein-protein interaction (PPI) engineering for drug discovery. Previous works have observed a high correlation between ΔΔ G and entropy, using probabilities of biologically important objects such as side chain angles and residue identities to estimate ΔΔ G. However, estimating the full conformational distribution of a protein complex is generally considered intractable. In this work, we propose a new approach to ΔΔ G prediction that avoids this issue by instead leveraging energy-based models for estimating the probability of a complex's conformation. Specifically, we novelly decompose ΔΔ G into a sequence-based component estimated by an inverse folding model and a structure-based component estimated by an energy model. This decomposition is made tractable by assuming equilibrium between the bound and unbound states, allowing us to simplify the estimation of degeneracies associated with each state. Unlike previous deep learning-based methods, our method incorporates an energy-based physical inductive bias by connecting the often-used sequence log-odds ratio-based approach to ΔΔ G prediction with a new ΔΔ E term grounded in statistical mechanics. We demonstrate superiority over existing state-of-the-art structure and sequence-based deep learning methods in ΔΔ G prediction and antibody optimization against SARS-CoV-2.
Beyond Fixed Variables: Expanding-variate Time Series Forecasting via Flat Scheme and Spatio-temporal Focal Learning
PDF ↗Multivariate Time Series Forecasting (MTSF) has long been a key research focus. Traditionally, these studies assume a fixed number of variables, but in real-world applications, Cyber-Physical Systems often expand as new sensors are deployed, increasing variables in MTSF. In light of this, we introduce a novel task, Expanding-variate Time Series Forecasting (EVTSF). This task presents unique challenges, specifically (1) handling inconsistent data shapes caused by adding new variables, and (2) addressing imbalanced spatio-temporal learning, where expanding variables have limited observed data due to the necessity for timely operation. To address these challenges, we propose STEV, a flexible spatio-temporal forecasting framework. STEV includes a new Flat Scheme to tackle the inconsistent data shape issue, which extends the graph-based spatio-temporal modeling architecture into 1D space by flattening the 2D samples along the variable dimension, making the model variable-scale-agnostic while still preserving dynamic spatial correlations through a holistic graph. Additionally, we introduce a novel Spatio-temporal Focal Learning strategy that incorporates a negative filter to resolve potential conflicts between contrastive learning and graph representation, and a focal contrastive loss as its core to guide the framework to focus on optimizing the expanding variables. To evaluate the effectiveness of STEV, we benchmark EVTSF performance on three real-world datasets from various domains and compare it against three potential solutions employing state-of-the-art (SOTA) MTSF models tailored for EVSTF. Experimental results show that STEV significantly outperforms its competitors, especially in handling expanding variables. Notably, STEV, with only 5% of observations during the expanding period, is on par with SOTA MTSF models trained with complete data. Further exploration of various expanding scenarios underscores the generalizability of STEV in real-world applications.
This paper studies grid-free point process modeling under varying fluid parameters. Existing methods rely on grid-based approaches or fixed parameters, making it challenging to handle complex nonlinear dynamics and out-of-distribution (OOD) scenarios. To address this, we propose Adaptive Perturbation Graph ODE (AGODE), a novel framework that integrates three key innovations: (1) an adaptive conditioning mechanism for physical parameter adaptation(2) a continuous graph neural ODE for spatiotemporal evolution modeling, and (3) a perturbation module with mutual information maximization for uncertainty quantification. AGODE employs graph neural networks to encode unstructured point cloud data into latent dynamics governed by neural ODEs, where physical parameters are injected through context-aware conditioning vectors. The perturbation module generates diverse trajectory samples by introducing stochastic noise during ODE integration, while contrastive learning aligns predictions with physical contexts to filter implausible outcomes. Extensive experiments across five fluid dynamics benchmarks (Prometheus, Navier-Stokes, Spherical-SWE, 3D Reaction-Diffusion, ERA5) demonstrate AGODE's state-of-the-art performance. Specifically, AGODE achieves MSE of 0.0302/0.0312 (in-domain/OOD) on Prometheus, outperforming PURE by 6.5%/5.0%, and reduces Navier-Stokes errors by 28.1% compared to physics-informed NMO. Notably, AGODE maintains superior OOD generalization with only 2.9% average error increase versus 7.8% for baselines, while its uncertainty quantification improves prediction reliability by 41% (95% confidence interval coverage). These results validate AGODE's capabilities in continuous spatiotemporal modeling, multi-parameter adaptation, and robust uncertainty estimation for complex fluid systems.
Weather forecasting is essential but remains computationally intensive and physically incomplete in traditional numerical weather prediction (NWP) methods. Deep learning (DL) models offer efficiency and accuracy but often ignore physical laws, limiting interpretability and generalization. We propose PhyDL-NWP, a physics-guided deep learning framework that integrates physical equations with latent force parameterization into data-driven models. It predicts weather variables from arbitrary spatiotemporal coordinates, computes physical terms via automatic differentiation, and uses a physics-informed loss to align predictions with governing dynamics. PhyDL-NWP enables resolution-free downscaling by modeling weather as a continuous function and fine-tunes pre-trained models with minimal overhead, achieving up to 170× faster inference with only 55K parameters. Experiments show that PhyDL-NWP improves both forecasting performance and physical consistency.
Existing work in physical-informed machine learning (PIML) has shown that data-driven learning of solution operators can provide a fast approximate alternative to classical numerical ordinary/partial differential equations (ODEs/PDEs) solvers. Of these, Neural Operators (NOs) have emerged as particularly promising. However, a key challenge in the field of NOs lies in developing methods that can effectively handle out-of-distribution (OOD) forecasting problems. Such problems involve the ability to adaptively learn from observations of the same dynamical system governed by ODEs/PDEs, where the underlying parameters are unknown and vary across instances. These tasks further require precise predictions even when faced with initial conditions and PDEs/ODEs parameters outside the training distribution. In this study, we consider the problem of training models in a risk-reverse manner. We introduce a risk-aware framework aimed at enhancing the OOD robustness of NOs by stochastically optimizing the conditional value-at-risk (CVAR) of a loss distribution. Through experiments on different distinct OOD tasks, our approach demonstrates a significant performance improvement over existing advanced NOs.
Current deep learning approaches for lithium-ion battery analysis are often specialized and limited to specific battery types or individual tasks. While recent advances in large language models (LLMs) highlight the potential of pretraining paradigms, existing time-series pretraining models inadequately address the physicochemical complexity and temporal irregularity inherent to battery operational data. We propose LiPM, a pretrained foundation model that unifies multi-dataset learning through physics-aware objectives and irregularity-tolerant temporal modeling. LiPM introduces three key innovations: (1) A Mix-Masked Autoencoder (MMAE) enforcing electrochemical consistency via joint reconstruction of temporally masked patches and cross-channel masked variables(2) A Coulombic Integration Regression (CIR) task explicitly encoding charge conservation laws, and (3) A dual-scale temporal encoder combining irregular intra-patch processing (preserving raw timestamps) with regular inter-patch attention (capturing macroscopic dynamics). Trained on eight heterogeneous battery datasets without cycle-label annotations, LiPM demonstrates universal applicability across partial charge-discharge segments and irregular sampling protocols. Extensive experiments show remarkable improvements over 9 state-of-the-art baselines in critical downstream tasks.