Multi-step retrosynthetic planning aims to decompose target molecules into available starting materials by iteratively invoking single-step prediction models within a search algorithm. The success of retrosynthetic planning depends on the joint guidance of single-step reasoning and global search across steps. However, most existing frameworks make step-wise decisions based only on the current molecular state, without explicitly modeling synthesizability signals that reflect long-range reachability. In this work, we propose GuideRetro, a synthesizability-aware framework for multi-step retrosynthetic planning that integrates global synthesizability knowledge into step-wise retrosynthetic prediction. GuideRetro learns transferable knowledge from large-scale reaction networks by modeling the evolution of synthetic complexity along reaction pathways. During planning, a route-aware synthesis state modeling module combines the evolving retrosynthetic route with retrieved global signals to guide reactant generation at each step. Experiments on benchmark datasets show that GuideRetro achieves state-of-the-art performance. The integration of global synthesizability knowledge and route-aware modeling improves planning accuracy and search efficiency under realistic retrosynthetic settings. The code is available at: https://github.com/L-AJ/GuideRetro.
论文检索
输入标题、作者或关键词，从 3,314 篇学术成果中精准定位
TaylorMoDe-GS: Taylor-Driven Gaussian Splatting Motion Model for Multi-View Dynamic Scene Deblurring
While 3D Gaussian Splatting (3DGS) has excelled in dynamic scene reconstruction, it struggles with multi-view object motion blur, where view-dependent non-uniform blur violates fundamental multi-view geometric constraints. Existing methods fail to balance complex motion fitting with physical consistency across different views. To address the challenge, we propose TaylorMoDe-GS, the first 3DGS framework tailored for multi-view dynamic object deblurring. Specifically, we shift the modeling paradigm from displacement fitting to velocity driven modeling. This is achieved by analytically deriving instantaneous 3D velocities via Taylor series. To map 3D physical motion onto the 2D image plane, we propose a velocity splatting technique. Building upon this, we introduce a neural Peano remainder network to compensate for high frequency non-linear dynamics, effectively resolving the conflict between physical priors and fitting flexibility. Combined with a learnable physical blur synthesis mechanism, our framework ensures rigorous spatiotemporal and view consistency. Extensive experimental results validate that our method achieves notable advancements in restoring high-fidelity dynamic scenes and physically consistent motion fields, effectively addressing the challenges of multi-view dynamic object deblurring. The code is released at https://github.com/Chiffin-0816/TaylorMoDe-GS.
Existing multi-modal automatic modulation recognition (AMR) methods primarily focus on exploiting multi-view representations of raw signal data to improve performance, but still struggle to effectively model and exploit high-level human prior knowledge. Although recent studies attempt to introduce large language models (LLMs) to integrate the textual modality, most of them merely treat LLMs as feature extractors, without further exploiting the LLM's potential. To this end, we propose a knowledge-enhanced multi-modal automatic modulation recognition framework based on large language model (AMR-LLM). Specifically, we first specially design an AMR instruction construction mechanism based on knowledge to activate the LLM's potential for signal perception, which designates the LLM as a signal domain expert, introduces human prior knowledge as a supplement, and exploits the unique physical information of each data sample as connection guidance. Then, to enable the LLM to directly perceive digital signals, we introduce a progressive multi-modal fusion strategy mapping signal features into the space of LLMs. Experimental results on multiple benchmark datasets demonstrate that AMR-LLM achieves approximately 5% higher accuracy than current state-of-the-art methods. Moreover, it achieves the current domain performance with only 30% of the data.
Spatio-temporal forecasting is inherently bottlenecked by a phenomenon we term Spatio-Temporal Duality: the paradoxical coexistence of time-varying physical lags and instantaneous semantic synchrony. Constrained by passive coupling, existing architectures often employ indiscriminate spatial aggregation that fails to dynamically arbitrate interaction intensity based on traffic states, forcing models to rely on redundant deep stacking to approximate complex dynamics. We address this issue by introducing Time Arbitrated Spatial GNN (TAS-GNN), an efficient framework that arbitrates spatial interactions through time. Our model leverages deep temporal semantics to dynamically and proactively manage the aggregation of spatial domains, effectively decoupling physical connectivity from semantic relevance. In addition, to ensure model simplicity, we also introduce spectral decoupling via discrete wavelet Transform (DWT) to capture multiscale dependencies with minimal overhead. Experiments on three real-world datasets (PEMS04, 07, 08) show that TAS-GNN not only outperforms the current baseline model in accuracy, but also significantly improves the inference speed while reducing the number of parameters.
In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, where workloads are highly dynamic and sensitive to runtime thread configurations. Existing multithreaded strategies struggle to match thread resources before or during execution, causing resource contention, scheduling overhead, and reduced throughput. Through empirical analysis, we identify the ratio of task execution time to scheduling time as the key factor determining the optimal thread count. Building on this insight, we propose AutoThread, a hybrid adaptive thread-tuning method for mitigating simulation bottlenecks in RL inference. AutoThread employs a Physics-Informed Neural Operator (PINO) as a thread-count predictor and incorporates a finite-source M/M/1 queueing model to constrain and guide prediction, enabling fast and accurate estimation under dynamic workloads. It further performs load-aware online fine-tuning to compensate for prediction errors and refine resource allocation. Experiments show that AutoThread improves average speedup by 18.4% over static strategies, achieves average throughput of 1.7× and 1.8× that of XGBoost and Reinforcer, respectively, and reduces execution time by up to 83.8% compared with state-of-the-art methods. Our code and dataset are publicly available at https://github.com/suchenjm/AutoThread.
Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with egocentric video to enable context-aware modeling of the driver's attention over time. Furthermore, to tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset that augments four existing driving datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for multimodal cognitive distraction detection. Our codes and CogDrive dataset resources are available at https://github.com/langzhang2000/EyeCue.
Vision-Language-Action (VLA) models bridge multimodal reasoning with physical control, but adapting them to new tasks with scarce demonstrations remains unreliable. While fine-tuned VLA policies often produce semantically plausible trajectories, failures often arise from unresolved geometric ambiguities, where near-miss actions lead to divergent execution outcomes under limited supervision. We study few-shot VLA adaptation from a generation-selection perspective and propose a novel framework, VGAS (Value-Guided Action-chunk Selection). It performs inference-time best-of-N selection to identify action chunks that are both semantically faithful and geometrically precise. Specifically, VGAS employs a finetuned VLA as a high-recall proposal generator and introduces Q-Chunk-Former, a geometrically grounded Transformer critic to resolve fine-grained geometric ambiguities. In addition, we propose Explicit Geometric Regularization (EGR), which shapes a discriminative value landscape to preserve action ranking resolution among near-miss candidates while mitigating value instability under scarce supervision. Experiments and theoretical analysis demonstrate that VGAS consistently improves success rates and robustness under limited demonstrations and distribution shifts. Our code is available at https://github.com/Jyugo-15/VGAS
Hand Object Interaction (HOI) generation provides an efficient solution for virtual reality simulation and embodied AI deployment.Recent studies have explored instruction-driven HOI synthesis, yet they overlooked the fine-grained interactive contact and struggled with robust generalization in data-scarce scenarios. In addition, existing datasets are frequently constrained with limited object categories and monotonous motion patterns. To address these limitations, we first integrate a large-scale HOI dataset, namely HOI-X, which contains diverse objects, rich motion patterns, and part-level contact annotations. Along with HOI-X, we propose A3fford-HOI, a fine-grained and generalizable Hand Object Interaction framework with Anatomy-Aligned Affordance Disentanglement. Specifically, we define a Disentangle-2-Interact paradigm in A3fford-HOI. Considering the various interactions of distinct hand anatomical structures, in the Disentangle stage, we devise an Anatomy-Aligned Affordance Predictor, which leverages the external plausible physical priors in 3D multimodal large language models for affordance disentanglement prediction. Then, the disentangled affordance is employed into the Interact stage, i.e. Affordance Driven Interactor. The interactor integrates textual instructions, object geometric features, and the external knowledge enhanced (MLLMs) affordance via a Diffusion Transformer (DiT) for generalizable HOI generation. Extensive experiments demonstrate that A3fford-HOI maintains superior generation quality and contact plausibility for fine-grained and generalizable hand object interaction.
Accurate traffic flow prediction is fundamental to Intelligent Transportation Systems (ITS). However, traffic dynamics exhibit inherent multi-scale heterogeneity, where stable global trends are often masked by stochastic local fluctuations. Existing methods struggle to reconcile these conflicting resolutions, leading to sub-optimal forecasting. To address this, we propose the Multi-Scale Spatial-Temporal Diffusion Transformer (MS-STDT). Deviating from previous diffusion approaches that treat generation as a monolithic task, we reframe the forward diffusion process as an intrinsic temporal coarse-graining operation. By leveraging the data's multi-scale hierarchy as a structural anchor, we introduce a coarse-to-fine latent guidance strategy that enables the model to reconstruct stable global trends before refining fine-grained details. This ensures physical consistency and generative stability without requiring external labels. Extensive experiments across six real-world datasets confirm that MS-STDT performs the best, demonstrating significant improvements in predictive accuracy and zero-shot robustness against sensor failures. We provide code and data at https://github.com/ZetaoLiPhD/MS-STDT.
Diffusion models show significant potential for low-light image enhancement. However, this task requires satisfying human perceptual preferences and content fidelity transcending simple brightness and color improvement. Existing methods rely on heuristic physical priors or incorporate perceptual metrics directly into timestep-wise training objectives. Such proxy constraints fail to provide reliable trajectory-level guidance for perceptual alignment. Furthermore, they often lead to artifacts or unnatural visual effects in this ill-posed inverse problem. To address these issues, we propose an unsupervised low-light enhancement framework based on Group Relative Policy Optimization (GRPO), which utilizes perceptual preferences to directly optimize the diffusion policy. We introduce a Sliding Window Hybrid ODE-SDE Sampling strategy that confines stochasticity to dynamic sub-intervals, thereby achieving efficient coarse-to-fine exploration and precise advantage attribution. To meet strict fidelity constraints, we construct a Synergistic Perception-Fidelity Reward and introduce an Independent Advantage Estimation strategy to mitigate signal collapse and gradient suppression in multi-objective optimization. Furthermore, we design a Temporal-Aware Dynamic Weighting Mechanism that adaptively adjusts perceptual weights across denoising stages to balance visual enhancement with structural preservation. Extensive experiments on multiple real-world benchmarks demonstrate that our method effectively improves the perceptual performance of existing models, yielding results that better align with human aesthetics.
While recent video generative models can synthesize high-fidelity videos, they struggle to portray plausible physical interactions and the resulting state transitions, a critical bottleneck for applications in robotics and VR/AR. To address this, we introduce a framework to generate a scalable synthetic dataset of controllable interactions. Our pipeline leverages a structured taxonomy and state-of-the-art image editing models to create explicit `start' and `end' state images, which serve as visual anchors for the interaction. To generate a seamless video utilizing these anchors, we propose State-Guided Sampling (SGS), a novel sampling technique that mitigates artifacts common in naive conditional generation. Furthermore, we develop and validate a new automated evaluation system that aligns with human judgments to ensure data quality. Experiments show that fine-tuning a base model on our dataset significantly enhances its ability to generate plausible interactions.
Hyperspectral image (HSI) clustering facilitates the unsupervised discrimination of complex surface materials but traditionally relies on the idealized assumption of centralized data availability. In real-world scenarios, however, this assumption clashes with data privacy regulations and the physical distribution of data across isolated silos. While Federated Learning offers a decentralized solution, existing frameworks in remote sensing are predominantly confined to supervised paradigms and struggle to address the heavy reliance on annotations and Non-IID distributions inherent among clients. To overcome these limitations, we propose a novel framework named Robust Federated HSI Clustering(RFHC) that enables collaborative unsupervised learning without requiring raw data exchange. Specifically, we design a dual-encoder architecture that incorporates a Federated Model Weight Augmentation strategy(FMWA), which generates consistent views through local-global network interactions to mitigate the spectral distortion introduced by traditional data augmentation. Furthermore, we develop a hybrid optimization objective that synergizes prototype relationship optimization with contrastive learning. This mechanism utilizes global prototypes to guide local training, effectively stabilizing feature learning against data heterogeneity while ensuring intra-cluster compactness. Extensive experiments on three benchmark HSI datasets demonstrate the effectiveness and superiority of the proposed method against state-of-the-art model(SOTA) federated approaches.
Elevating AI on the Edge: A Demonstration of MIMaaS (Machine Intelligence with Microcontroller-as-a-Service)
PDF ↗Deploying AI on microcontrollers (MCUs) is challenging. We introduce MIMaaS, a Microcontroller-as-a-Service platform that enables users to upload a model, select a target device, and receive a detailed performance report remotely. A key innovation is our measurement of real-world power consumption, alongside latency and memory usage, directly from the physical hardware. MIMaaS empowers researchers and developers to easily create and validate hardware-aware AI models without needing physical hardware access.
Large Language Models (LLMs) have revolutionized the simulation of agent societies, enabling autonomous planning, memory formation, and social interactions. However, existing frameworks often overlook systematic evaluations for event organization and lack visualized integration with physically grounded environments, limiting agents' ability to navigate spaces and interact with items realistically. We develop MiniAgentPro, a visualization platform featuring an intuitive map editor for customizing environments and a simulation player with smooth animations. Based on this tool, we introduce a comprehensive test set comprising eight diverse event scenarios with basic and hard variants to assess agents' ability. Evaluations using GPT-4o demonstrate strong performance in basic settings but highlight coordination challenges in hard variants.
We present MemoVision, a digital catalog system that captures semantic, spatial, temporal and interaction information as users move around physical environments using client devices such as smart glasses. The system utilizes open-vocabulary semantic segmentation and 3D scans to store objects-of-interest with comprehensive semantic, spatial, temporal and interaction labels. Our demonstration shows multimodal information query and retrieval capabilities, supporting specific queries about object locations, temporal events and user interactions including eye gaze and hand poses, enabling more contextualized responses compared to current multimodal large language models.
Industrial automation in smart logistics and factories requires simulation platforms that support rapid environment building before costly physical deployment. Yet existing tools often require substantial expertise, complex setup, and long configuration times, hindering agile prototyping. We present RAPID, a simulation platform with two components: layout design, which enables intuitive visual configuration of factory layouts, and behavior simulation and validation, which allows users to attach behavior models and evaluate system performance. RAPID lowers the entry barrier to industrial simulation, letting users apply existing behavior models or trained reinforcement learning (RL) agents to new layouts with minimal effort. This approach lets practitioners prototype facilities in minutes rather than weeks and gives researchers a standardized environment for benchmarking multi-agent RL and coordination algorithms. By combining rapid design with simulation-based validation, RAPID accelerates automation development from concept to implementation.
Physics Consistent World Models via Schrödinger-Bridge Optimal Transport for Computational Imaging and 3D-Consistent Video Generations
PDF ↗Modern generative models often violate basic physical principles. Shadows drift, geometry becomes inconsistent across views, and measurement models are ignored, which limits trust in both video synthesis and computational imaging. We propose a finite time Schrödinger Bridge (SB) world model that formulates generation as entropy regularized optimal transport from a simple prior to a distribution that is consistent with both data and physics. Instead of applying consistency corrections only at the final output, the framework introduces geometric and physical structure directly along the generative path. For video, the model enforces multiview geometric constraints through reprojection and epipolar agreement, homographies, and depth guided warping. For imaging, it incorporates differentiable optical operators, including point spread function based defocus models and lightweight Fourier propagation for coherent and partially coherent settings. When camera poses are known, the model penalizes reprojection error and warp aligned photometric or feature inconsistencies. When poses are unknown, a compact motion or flow estimator encourages cycle consistent trajectories. A lightweight UNet or Vision Transformer backbone, together with a short SB horizon, maintains computational efficiency. Evaluation will measure three dimensional and temporal consistency, physics fidelity through forward simulation residuals, and overall generative quality and efficiency using FID, KID, and FVD. Comparisons will include modern video diffusion models, plug and play data consistency methods, and unconstrained SB variants. The central hypothesis is that constraining the entire generative trajectory, rather than only the final frame, can shorten sampling while improving cross view coherence and physical plausibility across diverse sensing modalities, including cameras, microscopes, and medical imaging systems.
RNA 3D structure prediction is essential for understanding regulatory mechanisms, catalysis, and therapeutic RNA design, yet progress has lagged behind proteins due to limited structural data and the complexity of RNA folding. This work proposes a data-efficient, physics-informed deep learning framework for full atomistic prediction of transfer RNA (tRNA) tertiary structures directly from sequence. Our approach will integrate pretrained RNA embeddings, predicted secondary structure constraints, and SE(3)-equivariant graph attention to model long-range geometric relationships. A two-stage design will first predict global phosphate backbone coordinates, then reconstruct nucleobase atoms using a local geometry-aware decoder. A multi-objective loss will combine geometric accuracy with chemical and biophysical plausibility to enforce valid torsion angles, base-pairing, and steric constraints. We will benchmark against physics-based (VFold) and neural network–based (DeepFoldRNA) models to assess generalization under data scarcity. Ultimately, this framework aims to advance RNA 3D modeling with improved stability, interpretability, and capacity to generalize beyond well-characterized RNA families, supporting future applications in rational RNA engineering and structure-guided RNA design.
Spatiotemporal forecasting has seen remarkable progress with the advent of deep learning, particularly with Spatiotemporal Graph Neural Networks (STGNNs). These models excel at answering the what question: predicting future numerical values with high accuracy. However, they fail to answer the crucial why question. In high-stakes domains such as meteorology, urban planning, and public health, this opacity creates a critical bottleneck for adoption. A model that predicts a severe pollution event without explaining its atmospheric drivers is a black box, limiting its trustworthiness and utility for decision-makers who need actionable, causal insights. To address this critical gap, I propose a long-term research project to develop Causal-LLM, a new class of foundation models for spatiotemporal data that are both predictively powerful and causally interpretable. My central thesis is that genuine interpretability cannot be an afterthought; it must be designed into the model's core learning process. By adapting the powerful Time-LLM reprogramming framework and introducing a novel training methodology I term causal data synthesis, Causal-LLM will learn to not only forecast future states but also to articulate the human-understandable causal narratives behind them. This research will make two primary contributions: (1) a novel hybrid architecture that synergizes the perceptual power of GNNs with the reasoning capabilities of LLMs for complex physical systems, and (2) a new training paradigm that explicitly teaches this mapping. A successful project would provide a blueprint for a new class of trustworthy foundation models for science, enabling applications such as a climate model that not only predicts a flood but also explains the atmospheric river causing it, empowering authorities to make more informed and trusted decisions.
DINOv3-Powered Multi-Task Foundation Model for Quantitative Remote Sensing Estimation (Student Abstract)
PDF ↗Quantitative remote sensing estimation is critical for environmental monitoring, providing continuous measures of vegetation indices, canopy height, and carbon stock. Traditional radiative-transfer models and empirical regressions require expert knowledge and generalize poorly, while deep learning methods remain task-specific. We propose SatelliteCalculator+, a DINOv3-powered multi-task foundation model for continuous regression of spectral and structural variables. The framework combines prompt-driven cross-attentive adapters with lightweight MLP decoders, enabling efficient dense prediction from frozen features. To overcome limited supervision, we synthesize over one million paired samples from SPOT 6/7 imagery using physically defined formulas. On the Open-Canopy dataset, SatelliteCalculator+ achieves competitive accuracy across eight ecological variables while reducing inference cost, demonstrating the promise of self-supervised transformers and scalable multi-task learning for large-scale Earth observation.