Accurate identification of non-rigid motion is crucial for geometric validity in self-supervised monocular depth estimation (MDE), yet it remains challenging for current methods. Inspired by Human Visual Perception (HVP), we present VFM-Dynamo, an efficient self-supervised MDE framework that disambiguates non-rigid objects by combining a vision foundation model (VFM) such as Grounding DINO and SAM with coarse to fine motion identification strategy: 1) Integrating visual-textual feature dependencies, we activate the capability of Grounding DINO global understanding with text prompts to detect potential object bounding boxes. 2) We then estimate coarse motion mask from reprojection error and photometric consistency to separate static from dynamic content. This mask provides coarse guidance for constraining rigid flow in static regions through two analogous estimates aligned by a consistency loss, and it drives a non-dynamical suppression module that removes boxes associated with static objects. 3) The remaining boxes are used to prompt SAM to produce refined motion identification, which in turn guide the joint optimization of depth and optical flow. Additionally, due to foreground–background mixing, standard interpolation-based upsampling often produces boundary artifacts. We introduce a learnable neighbor affinity interpolation (LNAI) module, which directly upsamples depth to full resolution and can be seamlessly integrated as a plug-and-play component. Experiments on a series of benchmarks demonstrate that proposed framework achieves state-of-the-art performance among unsupervised methods. Code is available at https://github.com/Qianqian3764/VFM-Dynamo.
论文检索
输入标题、作者或关键词，从 100,903 篇学术成果中精准定位
Existing image deraining models struggle to handle complex real-world scenarios with rain-mist coupled degradation. The core challenge stems from the high visual similarity and spatial coupling between rain streaks and mist, making it difficult to accurately model their joint degradation patterns. Consequently, both specialized deraining models and all-in-one models for multiple degradations are ineffective in rain-mist coexistence scenarios. To address these challenges, we propose a novel Rain-Mist Removal (RMR) framework. It effectively utilizes single degradation priors from existing deraining and dehazing datasets to model the joint degradation, thereby achieving effective rain streak removal while preserving background structures obscured by mist. To enhance the generalization to real-world scenarios, we leverage text prompts trained in the CLIP perceptual space to drive the generated results toward real samples. Extensive experiments demonstrate that the proposed RMR outperforms state-of-the-art methods in rain-mist coexistence scenarios.
LLMs have shown immense potential for code translation, yet they often struggle to ensure both syntactic correctness and semantic consistency. While preference-based learning offers a promising alignment strategy, it is hindered by unreliable semantic rewards derived from sparse test cases or restrictive reference translations. We argue that a robust semantic reward for code translation must be derived directly from the source code. In this paper, we propose CTO to improve code translation with syntax-guided and semantic-aware preference optimization. Through contrastive learning, we train a cross-lingual semantic model to directly assess functional equivalence between source and translated code. By formulating code translation as a multi-objective optimization problem, this robust semantic signal is seamlessly unified with compiler-based syntactic feedback within the direct preference optimization framework. Extensive experiments on C++, Java, and Python translations demonstrate that CTO significantly outperforms existing baselines and alternative preference optimization strategies.
A central bottleneck in unsupervised cross-modal hashing is that both similarity guidance and objective weighting are typically static, while the semantic structure induced during training is inherently dynamic. We present DSSL-Hash, which turns semantic structure learning into an evolving process rather than a one-shot preprocessing step. Specifically, DSSL-Hash distills semantic cues from a vision–language foundation model and constructs a dynamic perception graph via an adaptive similarity matrix, enabling graph propagation to capture higher-order relations beyond pairwise matching. We further reshape Hamming-space learning with a semantic-channel constraint that explicitly regulates distances for partially similar pairs, reducing semantic distortion after binarization. Finally, we develop a Retrieval-Aware Adaptive Scheduler (RAAS) that leverages retrieval feedback to co-adjust modality interactions and objective weights, achieving robust optimization without extensive manual hyper-parameter search. Extensive experiments on MS COCO, NUS-WIDE, and MIRFLICKR-25K across multiple code lengths demonstrate that DSSL-Hash consistently outperforms recent state-of-the-art unsupervised cross-modal hashing methods.
The Segment Anything Model (SAM) provides a new paradigm for visual perception tasks through large-scale pre-training and interactive prompts. However, the quality of mask generation is limited by the difficulty of aligning the optimization direction between the backbone and the prompt, which is especially significant in non-natural domain images with low contrast and indistinct semantic edge structure features, especially in SAR images. Therefore, this paper proposes a coupling optimization mechanism that integrates the originally independent prompt generation process with the backbone into a closed loop through the Point Impact Decomposition (PID) module to guide the iterative optimization of the prompt. Specifically, this paper introduces the concept of PID in the decoder to quantify the contribution of prompt points to the masks, constructing an interpretable reward variable to drive the reinforcement learning prompt optimizer. This optimizer designs a reward function based on hypothesis testing ideas, achieving iterative updates of prompt points, and providing feedback on the effects of mask generation based on prompt points. Experiments show that this mechanism can provide a prompt that fits image features; its performance exceeds the existing baseline, which effectively improves the generalization ability of SAM on SAR images.Code is available at https://github.com/Fm336/PID-SAM.
Estimating object mass from visual input is challenging because mass depends jointly on geometric volume and material-dependent density, neither of which is directly observable from RGB appearance. Consequently, mass prediction from pixels is ill-posed and therefore benefits from physically meaningful representations to constrain the space of plausible solutions. We propose a physically structured framework for single-image mass estimation that addresses this ambiguity by aligning visual cues with the physical factors governing mass. From a single RGB image, we recover object-centric three-dimensional geometry via monocular depth estimation to inform volume and extract coarse material semantics using a vision-language model to guide density-related reasoning. These geometry, semantic, and appearance representations are fused through an instance-adaptive gating mechanism, and two physically guided latent factors (volume- and density-related) are predicted through separate regression heads under mass-only supervision. Experiments on image2mass and ABO-500 show that the proposed method consistently outperforms state-of-the-art methods.
Predicting molecular spectra requires modeling 3D conformation and solvent modulation. However, E(3)-equivariant networks based on local message passing exhibit limited sensitivity to long-range geometric dependencies, affecting the discrimination of globally distinct conformers. We introduce the Multiscale Geometric Interaction Layer (MGIL), which integrates global context by augmenting local features with centroid-referenced anchors, geometric moments, and virtual nodes. This design explicitly encodes global anisotropy while maintaining equivariance. Furthermore, we propose a Solvent Field Modulator (SFM) to encode solvent topology for conditional feature adaptation. Experiments demonstrate that MGIL enhances the capture of global structural variations, yielding consistent performance gains across spectral prediction benchmarks while maintaining linear computational efficiency.
Large language models have shown promising potential in chemistry, with prior work exploring molecular recognition, classification, and property prediction. Despite the achieved progress, LLMs are still far from satisfactory when dealing with complex chemical multi-step reaction pathway inference task due to the lack of domain knowledge and limited ability to maintain consistent multi-step reasoning. To address these challenges, we propose ChemKGL, a novel multi-step reasoning framework enhanced by knowledge graph retrieval. We reformulate the task as a knowledge graph retrieval problem to better control LLM generation, and leverage graph technologies to integrate textual descriptions, reaction conditions, and materials. Based on this reformulation, we design a novel multi-retrieval strategy to improve the performance of reaction pathway inference, which includes forward retrieval, reverse retrieval, and historical retrieval. An entropy-based pruning mechanism is further introduced to alleviate cold-start issues caused by retrieval failure. In addition, we construct a comprehensive chemical knowledge graph and corresponding dataset to support model training and analysis. Finally, we employ multiple evaluation methods to assess our proposed ChemKGL, and the experimental results demonstrate its superiority. Our code and dataset can be obtained at https://github.com/Double-Sail/ChemKGL.
The three-dimensional bin packing problem (3D-BPP) is widely applied in logistics and warehousing. However, existing learning-based approaches often neglect stability constraints and struggle to generalize across diverse bin dimensions. To address this, we propose a novel deep reinforcement learning framework, One4Many-StablePacker (O4M-SP). The primary advantage of O4M-SP lies in its ability to handle variable bin dimensions in a single training process while explicitly enforcing two types of practical stability constraints: support constraints, which ensure an item's bottom center lies within the convex hull of the underlying contact area, and weight constraints, which restrict the total vertical load on an item to its bearing capacity. Our training method introduces two innovative mechanisms. First, a weighted reward function integrates the loading rate with a novel height difference metric for packing layouts, promoting improved bin utilization via flatter packing configurations. Second, clipped policy gradient optimization with tailored policy drifting mitigates entropy collapse, encouraging exploration at critical decision nodes during packing to prevent premature convergence. Extensive experiments demonstrate that O4M-SP generalizes effectively across diverse bin dimensions and significantly outperforms baseline methods. Furthermore, the framework exhibits strong practical applicability by solving complex scenarios under strict stability constraints.
Capturing global context through attention is essential for reasoning over knowledge graphs, especially when relevant entities are distant or disconnected. To scale attention to large graphs, recent methods replace Softmax with kernel feature mappings, reducing computational complexity to linear in the number of nodes. While efficient, these approximations tend to produce overly smooth attention scores, which can reduce discrimination between correct triplets and hard negative samples. Moreover, they exhibit scale-dependent shifts in attention entropy, making them sensitive to changes in graph size during inductive inference. In this paper, we introduce LaGR, a novel approach for integrating global information in knowledge graph reasoning. Rather than approximating interactions among all nodes, LaGR compresses the graph into a fixed, compact set of latent summaries and applies exact self-attention within this latent space. This change yields scale-invariant attention and stable performance across diverse data settings. In addition, we propose a node-adaptive residual fusion mechanism that dynamically balances local and global information at the node level, leading to more expressive representations. Extensive experiments on both transductive and inductive benchmarks show that LaGR substantially outperforms state-of-the-art baselines, demonstrating that exact attention over latent summaries is an efficient and effective way to capture global context in knowledge graph reasoning. Our implementation is available at https://github.com/XMU-KG/LaGR.
Decision-focused learning has shown great promise for addressing predict-then-optimize problems, particularly in the presence of under-specified models. However, its practical deployment is often hindered by high computational costs and limited scalability, as it requires solving a constrained optimization problem for each training instance at every iteration. To address these challenges, we propose a novel framework that incorporates Lagrangian decomposition into the decision-focused learning paradigm. Specifically, we introduce a new surrogate objective along with two loss functions for evaluating and training the underlying prediction model. We further propose two variants of our approach, which offer different trade-offs between computational efficiency and solution quality. Our framework can be seamlessly integrated with standard decision-focused learning methods, including Smart Predict-then-Optimize (SPO+) and Implicit Maximum Likelihood Estimation (IMLE). Through experiments on two standard benchmarks, the multi-dimensional knapsack problem and quadratic portfolio optimization, we demonstrate that our approach achieves competitive performance while remaining amenable to parallelization. In particular, it consistently outperforms traditional decision-focused learning methods on large-scale instances, involving up to eight times more variables than those typically considered in related work. The implementation is available at https://github.com/corail-research/DFL-LD.
Multi-object tracking (MOT) has long faced challenges with identity switches, especially for targets with low appearance discriminability and complex motion. Existing end-to-end trackers typically enhance robustness by modeling long-term temporal information across target-level representations, yet these representations remain insufficiently discriminative for spatially and visually similar targets. In this paper, we present MoTRa, a Motion-aware Target Representation learning framework for end-to-end MOT that enriches target-level representations with adaptive motion cues. As a result, each representation adaptively balances motion cues and appearance-related content cues under varying conditions, enhancing its discriminability for tracking. To achieve this, we propose a feature fusion and alignment module that extracts motion cues and adaptively fuses them with content features into target representations. To further regularize the dynamically fused representations, we propose a target-specific contrastive learning strategy that promotes intra-trajectory consistency and inter-target separability. Experimental results demonstrate the effectiveness of our method, achieving competitive performance on multiple benchmarks.
Parameter-efficient adaptation methods, such as adapters and prompt learning, have become popular for transferring pre-trained vision-language models (VLMs) to downstream tasks. However, under limited supervision during adaptation, overly aggressive cross-modal alignment can distort the intrinsic structure of modality-specific representations, leading to degraded generalization. In this paper, we propose Co-adaptive Dual-path Alignment (CoDA), a new adaptation framework that explicitly disentangles and coordinates cross-modal semantic alignment and intra-modal structural consistency. In CoDA, we also propose a parent class labeling strategy that injects hierarchical semantic priors into textual prompts, further stabilizing alignment under limited supervision. Extensive experiments on eleven benchmark datasets show that CoDA outperforms state-of-the-art parameter-efficient methods, particularly under few-shot learning and distribution-shift scenarios. Our code is available at https://github.com/yizhang-ac/CoDA.
3D molecular linker design is a critical task in structure-based drug discovery, which requires the precise synthesis of chemical bridges to connect fragments within the constrained environment of a protein binding pocket. Existing methods often suffer from environmental blindness, treating the inter-fragment space as a vacuum and yielding candidates with poor binding affinity or severe steric clashes. To address this, we propose LinkerBridge, an equivariant framework that unifies biochemical semantics with physical constraints through two innovations: a Contextual Interaction-Aware Representation module that internalizes pre-existing biochemical semantics, and a Differentiable Physical Guidance mechanism derived from Van der Waals potentials to steer generation away from collision zones. Extensive evaluations on ZINC, GEOM, and BindingMOAD benchmarks, as well as Hsp90 and JNK3 case studies, demonstrate that LinkerBridge significantly outperforms state-of-the-art methods in real-world drug discovery.
Human experts often struggle to select the best option from a large set of items with multiple competing objectives, a process bottlenecked by the difficulty of formalizing complex, implicit preferences. To address this, we introduce LISTEN (LLM-based Iterative Selection with Trade-off Evaluation from Natural-language), an agentic LLM-based framework that treats the LLM as a decision-making agent capable of iteratively refining its internal preference model and taking actions (e.g., proposing utilities or selecting candidates) to maximize alignment with a user's implicit goals. To operate within LLM constraints like context windows and inference costs, we propose two iterative algorithms: LISTEN-U, which uses the LLM to refine a parametric utility function, and LISTEN-T, a non-parametric method that performs tournament-style selections over small batches of solutions. Evaluated on diverse tasks including flight booking, shopping, and exam scheduling, our results show LISTEN-U excels when preferences are parametrically aligned (a property we measure with a novel concordance metric), while LISTEN-T offers more robust performance overall. This work explores a promising direction for steering complex multi-objective decisions directly with natural language, reducing the cognitive burden of traditional preference elicitation. Code is available at https://github.com/AdamJovine/LISTEN.
Driver gaze estimation serves as a fundamental metric for evaluating driver attentiveness in modern monitoring systems. Beyond being vulnerable to sudden lighting changes and sensor noise, spatial-domain models struggle to disentangle authentic gaze cues from irrelevant visual attributes. In this paper, we propose LISA, a Language-guided Interference-aware Spatial-Frequency Attention framework that combines frequency-domain priors with vision-language knowledge. Observing that the amplitude spectrum remains relatively stable even under spatial perturbations, we design a dual-domain fusion mechanism. It integrates stable low-frequency semantics into high-frequency details, employing spatial attention to precisely target ocular regions. To reduce semantic ambiguity, we also introduce a training-time disentanglement strategy. Using a frozen CLIP encoder and orthogonal regularization, we explicitly separate gaze features from appearance interference. Experiments on two benchmarks show that LISA achieves state-of-the-art performance, with significantly improved robustness against occlusions and lighting variations. The code repository is available at https://github.com/Mason-bupt/LISA.
Satisfiability (SAT) with cardinality constraints arises naturally in many practical applications, where high-level counting requirements coexist with standard Conjunctive Normal Form (CNF) clauses. Translating these constraints into CNF can destroy structural information, limiting the effectiveness of search-based heuristics. In this paper, we propose a cardinality-aware local search framework to solve this problem, denoted as CardSAT-LS. CardSAT-LS integrates a preprocessing phase based on the generalized unit propagation and resolution, a cardinality-sensitive scoring function combining the make-break mechanism and cardinality violation, and an initialization based on fake-backbone variables. Furthermore, CardSAT-LS employs a unified framework that adaptively alternates between flip and swap operators when the search gets trapped in local optima. Finally, we conduct experiments on five public benchmarks from real-world applications as well as the MaxSAT and SAT competitions. Compared with ten state-of-the-art competitors, including SAT, MaxSAT, and PB solvers, CardSAT-LS solves the most instances with the lowest PAR-2 score. Additionally, we integrate CardSAT-LS into exact solvers for phase selection, which leads to significant speedups.
Movie dubbing is the task of synthesizing speech from scripts conditioned on video scenes, requiring accurate lip sync, faithful timbre transfer, and proper modeling of character identity and emotion. However, existing methods face two major limitations: (1) high-quality multimodal dubbing datasets are limited in scale, suffer from high word error rates, contain sparse annotations, rely on costly manual labeling, and are restricted to monologue scenes, all of which hinder effective model training; (2) existing dubbing models rely solely on the lip region to learn audio-visual alignment, which limits their applicability to complex live-action cinematic scenes, and exhibit suboptimal performance in lip sync, speech quality, and emotional expressiveness. To address these issues, we propose FunCineForge, which comprises an end-to-end production pipeline for large-scale dubbing datasets and an MLLM-based dubbing model designed for diverse cinematic scenes. The pipeline enables the construction of the first television dubbing dataset, CineDub, which serves as a high-quality foundation for training and evaluation. Building on this, our dubbing model effectively captures multimodal cues and supports complex dubbing scenarios, including monologue, narration, dialogue, and multi-speaker settings. Experiments demonstrate that our approach consistently outperforms state-of-the-art methods in audio quality, word error rate, lip sync, temporal alignment, timbre transfer, and instruction following. Code and demos are available at https://funcineforge.github.io/.
Large language models often face challenges in low-resource agglutinative language text summarization tasks due to poorly designed prompts, leading to core information dilution, reduced fidelity, and critical information loss caused by the complex grammatical structures of agglutinative languages. For traditional Mongolian, a typical low-resource agglutinative language, this paper proposes a Nucleus-to-Periphery Retrieval-Iterative Prompting (NPRIP). This method first guides the model to extract highly condensed semantic core information (events, persons, time, etc.) from the original text. Subsequently, through multiple rounds of self-refinement iteration, it progressively expands peripheral details (background, causes, consequences, secondary facts, etc.). The model performs fact consistency checks, redundancy removal, and fidelity correction on the current draft, achieving gradual improvements in information completeness and fidelity. To enhance Mongolian language representation, we perform parameter-efficient fine-tuning on LLaMA3-8B using the CCMT2019 Mongolian-Chinese parallel corpus, and construct a larger abstractive Mongolian news summarization dataset MoSum along with its augmented version. Experiments on traditional Mongolian text summarization tasks demonstrate that our proposed method significantly outperforms multiple baseline models on automatic evaluation metrics including ROUGE-1, ROUGE-2, and ROUGE-L. This validates the effectiveness of core-priority structured iterative prompting in low-resource agglutinative language summarization scenarios.
4DVarGen: A 4D Variational-Inspired Generative Model for Eddy-Resolving Surface Ocean Reconstruction
Sea surface variable reconstruction from sparse observations is a key ocean-science challenge. Traditional methods, such as the four-dimensional variational (4DVar) approach, rely on numerical models for background information, leading to high computational costs. Deep learning methods are more efficient but often fail to capture eddy dynamics, resulting in limited effective resolution. We propose 4DVarGen, a 4DVar-inspired generative framework for reconstructing sea surface variable fields at eddy-resolving scales from sparse remote-sensing observations. 4DVarGen establishes a mathematical equivalence between 4DVar and an observation-guided denoising process. Its key innovation is injecting the observation-likelihood gradient into denoising iterations, driving the generated trajectories to evolve in the direction of minimizing the 4DVar objective function toward a maximum a posteriori solution. Spatiotemporal priors learned by a diffusion model serve as background information, reducing computational costs and mitigating the adverse effects of Gaussian assumptions. Experiments show that 4DVarGen effectively leverages the temporal evolution patterns of sea surface temperature (SST) and sea surface height (SSH), as well as their dynamical mappings learned by the diffusion model, leading to improved reconstruction accuracy and effective resolution. Our model, pretrained on GLORYS12V1 reanalysis data, generates sea surface variable fields guided by real observations, achieving accuracy and effective resolution improvements of 18% and 58%, respectively, compared to GLORYS12V1. This study offers a novel framework for reconstructing Earth system states from sparse observations.