论文检索

Deep Learning · Foundation Models

jing wu, Jianhua Wu, Jiayi Guan, Jiahong Chen, Jinghui Lu, Hangjun Ye, Bingzhao Gao, Long Chen

Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D priors or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient \textit{\textbf{Spatio}-vision \textbf{L}anguage \textbf{M}odels (SpatioLM)}, that enhances spatial intelligence without extra 3D priors or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while maintains the general-purpose capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks.

Deep Learning · Large Language Models

Liang Zheng, Bowen Shi, Yitao Hu, Jiawei Zhang, Ruofan Li, Guotao Yang, Zhixin Zhao, Zhengchao Wang, Sheng Chen, Wenxin Li 等

Diffusion-based large language models (dLLMs) have emerged as a promising alternative to autoregressive models, leveraging simultaneous denoising to enable global planning and iterative refinement. These properties make dLLMs particularly attractive for long-context generation. However, deploying dLLMs faces a prohibitive memory capacity barrier, as existing inference systems are inefficient for the diffusion paradigm. We observe that current inference systems are misaligned with dLLMs. Unlike autoregressive models, whose memory footprint is dominated by the cumulative KV-Cache, dLLMs are bottlenecked by transient activations rematerialized per step. Moreover, generic memory reuse mechanisms lack the global visibility to handle dynamic memory peaks of dLLMs, which alternate between logits and feed-forward networks. To address these challenges, we present Mosaic, a memory-efficient inference system that shifts dLLM execution from local, static memory management to a global, dynamic paradigm. Mosaic integrates (i) a mask-only logits kernel eliminating redundant activation materialization, (ii) a lazy chunking optimizer using online heuristics to adaptively tame dynamic memory peaks, and (iii) a global memory manager leveraging virtual addressing to mitigate memory fragmentation. Extensive evaluations show that Mosaic reduces the memory peak-to-average ratio by 2.71$\times$ on average and increases the maximum supportable inference sequence length on identical hardware by 15.30--32.34$\times$. Crucially, Mosaic is training-free and preserves exact model outputs, while simultaneously reducing end-to-end latency by 2.5\%--55.4\%.

Ashwinkumar Badanidiyuru

Online advertising platforms rely on machine learning models to predict click-through rates (pCTR) and conversion rates (pCVR) for auction mechanisms. We introduce a novel framework to study the interaction between recommender system model quality, auction format, and au- tobidder behavior. We formalize when model improvements—defined via a refinement relation inspired by filtrations in probability theory—lead to improvements in platform-level Evaluation Cri- teria Metrics (ECM) such as revenue, welfare, or liquid welfare. Our main contributions are: (1) a formal definition of model improvement based on cluster refinement, and (2) a complete charac- terization of ECM monotonicity across different combinations of bidder types (tCPA, max-CPA), auction formats (first-price, second-price, VCG), and budget constraints. We show that first-price auctions with uniform bidding guarantee revenue monotonicity for tCPA bidders without budgets (via Jensen’s inequality), while second-price auc- tions and budget constraints can break this prop- erty. We provide full numerical counterexamples for all negative results. Our findings have practi- cal implications for advertising platforms seeking to align model improvements with business out- comes.

Deep Learning · Large Language Models

Dong-Hee Kim, Reuben Tan, DONGHYUN KIM

Visual agents employ tools such as zoom-in cropping within visual chains of thought to access fine-grained details. Prior work has primarily demonstrated the effectiveness of these tools on visual search tasks, leaving their applicability to more diverse and complex visual problems underexplored. In this paper, we move beyond visual search and study challenging visual tasks that require advanced spatial understanding and reasoning, such as 3D spatial reasoning, where agents must not only crop or zoom in on relevant regions but also understand how these local details relate to the global context. We identify a tool-use collapse phenomenon: models progressively stop using tools while still achieving higher task accuracy. Moreover, we observe a clear asymmetry: (i) completely eliminating tool use degrades performance, whereas (ii) incentivizing tool use yields only marginal gains despite substantially increasing usage. We find that vanilla training and rewarding tool-use encouragement reduce rollout diversity during training, explaining why higher tool-use does not yield stronger reasoning performance. Motivated by these findings, we encourage diverse rollout exploration by adding an entropy-regularization term to the reinforcement learning objective, which results in the best performance despite tool usage gradually declining during training. Overall, our findings suggest a training-time view of tools as scaffolding, where broader exploration in both text and vision shapes representations that improve despite tool-use collapse.

Deep Learning · Graph Neural Networks

Chendi Qian, Christopher Morris

Semidefinite programs (SDPs) are a powerful framework for convex optimization and for constructing strong relaxations of hard combinatorial problems. However, solving large SDPs can be computationally expensive, motivating the use of machine learning models as fast computational surrogates. Graph neural networks (GNNs) are a natural candidate in this setting due to their sparsity-awareness and ability to model variable-constraint interactions. In this work, we study what expressive power is sufficient to recover optimal SDP solutions. We first prove negative results showing that standard GNN architectures fail on recovering linear SDP solutions. We then identify a more expressive architecture that captures the key structure of SDPs and can, in particular, emulate the updates of a standard first-order solver. Empirically, on both synthetic and SDPLIB benchmarks of various classes of SDPs, this more expressive architecture achieves consistently lower prediction error and objective gap than theoretically weaker baselines. Finally, using the learned high-quality predictions to warm-start the first-order solver yields practical speedups of up to 80%.

Deep Learning · Large Language Models

Bo Zhang, Jinfeng Zhou, Yuxuan Chen, Jianing Yin, Minlie Huang, Hongning Wang

Large Language Models (LLMs) have shown significant potential in scientific discovery but struggle to bridge the gap between theoretical reasoning and verifiable physical simulation. Existing solutions operate in a passive "execute-then-response" loop and thus lack runtime perception, obscuring agents to transient anomalies (e.g., numerical instability or diverging oscillations). To address this limitation, we propose EmbodiedAct, a framework that transforms established scientific software into active embodied agents by grounding LLMs in embodied actions with a tight perception-execution loop. We instantiate EmbodiedAct within MATLAB and evaluate it on complex engineering design and scientific modeling tasks. Extensive experiments show that EmbodiedAct significantly outperforms existing baselines, achieving SOTA performance by ensuring satisfactory reliability and stability in long-horizon simulations and enhanced accuracy in scientific modeling.

Deep Learning · Graph Neural Networks

Youqing Wang, Jiahao Long, Tianxiang Zhao, Man Cao, Mengyuan Xin, Jiapu Wang, Junbin Gao, Jipeng Guo

Graph neural networks (GNNs) have been demonstrated to be powerful tools for analyzing structural graph data. However, most existing methods usually rely on fixed adjacency structures for information propagation, lacking strong adaptability to the latent semantic relationships that exist but are not explicitly connected in graph, especially in complementary high-pass and low-pass filtering views. To this end, this paper proposes a novel Dual-channel Dynamic Graph Neural Network (DCD-GNN), mainly consisting of parallel representation learning channels: a static structure-preserving channel and a dynamic adjacency-enhancing channel. The dynamic channel exploits both low-pass structural filtering and high-pass personalized detail via self-attention adjacency learning and then integrates them for comprehensive semantic modeling, while the static channel maintains structural stability. Both channels employ a multi-scale representation fusion mechanism and are finally fused into a unified and discriminative node embedding representation. Extensive experiments on various graph benchmark datasets verify the superiority of DCD-GNN in discriminative graph representation learning.

Applications · Computer Vision

Zhenhua Lei, Zefang Han, yu qiu

Scene graph generation (SGG) aims to parse an image into a structured graph of objects and their predicates, enabling explicit relational reasoning for visual understanding. However, prevailing methods often over-predict geometric predicates, resulting in scene graphs that are factually correct yet semantically shallow. While recent works effectively attribute this phenomenon to the long-tailed data distribution, we identify another critical factor driving such biased prediction: co-occurrence-induced representation entanglement, where geometric and non-geometric predicates that frequently co-occur are encoded into overly similar representations. To this end, we introduce Dual-stream Synergistic Network (DS-Net) that models geometric and non-geometric predicates with two specialized streams, coupled with a bidirectional cross-stream fusion mechanism. The space stream focuses on spatial and structural cues, while the vision stream captures fine-grained visual evidence and semantic priors. Extensive experiments show that DS-Net consistently improves predicate inference, achieving 1.3\% $\sim$ 6.1\% absolute gains in mR@100 on the SGGen task when integrated into existing SGG methods. These results highlight the importance of synergistic modeling of geometric and non-geometric predicates for generating semantically richer scene graphs.

Applications · Computer Vision

Luke Miller, Yugyung Lee

Segmenting small and sparse structures in large-scale images is fundamentally constrained by pixel-level, lattice-bound computation and extreme class imbalance--dense, full-resolution inference scales poorly and forces most pipelines to rely on fixed regionization or downsampling, coupling computational cost to image resolution and attenuating boundary evidence precisely where minority structures are most informative. We introduce **SEMIR** (*Semantic Minor-Induced Representation Learning*), a representation framework that decouples inference from the native grid by learning a task-adapted inference space. **SEMIR** transforms the underlying grid graph into a compact, boundary-aligned graph minor through parameterized edge contraction, node deletion, and edge deletion, while preserving an exact lifting map from minor predictions to lattice labels. Minor construction is formalized as a few-shot optimization problem that replaces hand-tuned preprocessing with a *boundary-alignment objective*: minor parameters are learned by maximizing agreement between predicted boundary elements and class-agnostic semantic edges under a *boundary Dice criterion*, and the induced minor is annotated with scale- and rotation-robust geometric and intensity descriptors and supports efficient region-level inference via message passing on a graph neural network (GNN) with relational edge features. We benchmark **SEMIR** on three tumor segmentation datasets—**BraTS2021**, **KiTS2023**, and **LiTS2017**—where targets exhibit high structural variability and distributional uncertainty, providing a stringent testbed for *structure-adaptive inference*. **SEMIR** yields consistent improvements in *minority-structure Dice* at practical runtime, positioning *minor-induced representations* as a principled alternative to pixel-centric segmentation in challenging, high-variability visual domains.

Deep Learning · Graph Neural Networks

Yao Cheng, Siqiang Luo

Relational deep learning (RDL) converts relational databases (RDBs) into heterogeneous graphs, but graphs derived directly from database schemas are often not well suited for how graph neural networks (GNNs) perform relational reasoning. We study what makes a relational graph suitable for deep learning and show that schema-derived graphs suffer from two systematic failures: information overload and semantic fragmentation. Through an empirical analysis on real-world databases, we find that effective graphs arise from a task-dependent balance between removing task-irrelevant structure and injecting task-aligned relational connectivity. Filtering exhibits a non-monotonic effect on performance, while structural injection is beneficial only when it reflects the logic of the downstream task. Based on these findings, we develop an end-to-end structural optimizer that applies both operations to adapt relational graphs automatically. Across 23 tasks spanning classification, regression, and recommendation, the optimized graphs consistently improve accuracy while often reducing inference cost. Code and data are available at https://anonymous.4open.science/r/Structural_Optimizer_RDL-0F74/.

Deep Learning · Graph Neural Networks

Jiaming Zhuo, Ziyi Ma, Kun Fu, Di Jin, Chuan Wang, Zhen Wang, Xiaochun Cao, Huazhu Fu, Liang Yang

By synergizing graph topology with the global expressive power of the attention mechanism, Graph Transformers (GTs) have emerged as a dominant architecture for node classification. However, existing models primarily focus on diverse topology injection mechanisms, specifically score-level and representation-level designs, yet lack a unified theoretical foundation to characterize how these mechanisms shape the representation propagation. To bridge this research gap, this paper unifies these designs under a common Graph Signal Denoising framework, revealing that denoising efficacy (\textit{i.e.}, representation quality) is fundamentally dictated by the block-diagonal structure of the propagation operator. To instantiate this prior efficiently, this paper introduces a novel Block-Diagonal GT architecture, named \textsc{BDFormer}, which enforces a block-diagonal constraint via spectral-regularized cross-attention on latent anchors. Specifically, by routing global interactions through these anchors, \textsc{BDFormer} imposes the spectral block-constraint directly on the anchor-level affinity. Crucially, the learned global affinity guides the pruning of local heterophilous edges, ensuring that both scales synergistically adhere to the target distribution. Extensive evaluations on benchmark datasets demonstrate the scalability and robustness of \textsc{BDFormer}.

Deep Learning · Graph Neural Networks

Kaixuan Yao, Ting Guo, Ming Li, Feilong Cao

In recent years, hypergraph contrastive learning methods have gained widespread attention due to their excellent performance in processing high-order structural data. However, traditional hypergraph learning method often assume that neighboring nodes are homogeneous, which can lead to the mixing of heterogeneous information in highly heterogeneous datasets, thereby affecting node feature representation. To address this issue, this paper proposes a heterogeneity-sensitive hypergraph contrastive learning method. In the view enhancement stage, we introduce a heterogeneity-aware mechanism that masks high-heterogeneity nodes using hyperedges as intermediaries for information filtering. This mechanism weakens the interference of heterogeneous nodes on view consistency, enabling the model to focus more on key features. In the encoding stage, a heterogeneity-sensitive hypergraph encoder is designed. It dynamically adjusts the weights of information propagation through hyperedges in two phases: ``node-to-hyperedge" and ``hyperedge-to-node". This adjustment allows hyperedges to focus on homogeneous information and feedback the aggregated homogeneous information to the respective nodes. Besides, we provide a theoretical proof that our model is capable of aggregating information based on node heterogeneity using hyperedges as intermediate structures. Extensive experimental results demonstrate that this method effectively reduces the interference of heterogeneous information and improves model performance on multiple benchmark datasets. Our code is availabl at: https://anonymous.4open.science/r/HHCL-F926

General Machine Learning · Transfer, Multitask and Meta-learning

Shaoran Lv, Xinyao Li, Jingjing Li

Test-time adaptation (TTA) adapts pretrained models to test data on-the-fly. Current TTA methods have focused on what to adapt: lightweight domain-aware components (prompts, normalization statistics) updated with consistency-aware self-supervised losses. This work investigates the more fundamental yet underexplored optimization process, providing insights and guidelines on how to appropriately update models for TTA. By analyzing the optimization error during TTA, we identify a pivotal stability-plasticity trade-off: the model should adapt to novel distributions while retaining learned knowledge, which motivates our design of a CONfidence-and-Gradient-Aware scheduler (CONGA) to constrain model learning rate (LR) within an adaptive exploration interval. For each iteration, the lower bound encourages model exploration on informative confident samples, while the upper bound prevents aggressive overfitting to noisy optimization gradients. Based on our theoretical findings, an adaptation-progress-conditioned cosine decay function decides the specific LR within the interval. As an LR scheduler, CONGA is naturally applicable on existing TTA methods as a plug-in module, introducing little computation overheads. Extensive experiments and analysis demonstrate the superiority and validness of CONGA.

Deep Learning · Graph Neural Networks

Jialiang Wang, Hanmo Liu, Shimin Di, Zhili Wang, Jiachuan Wang, Lei Chen, Xiaofang Zhou

Designing high-performance neural networks for new tasks requires balancing optimization quality with search efficiency. Current methods fail to achieve this balance: neural architectural search is computationally expensive, while model retrieval often yields suboptimal static checkpoints. To resolve this dilemma, we model the performance gains induced by fine-grained architectural modifications as edit-effect evidence and build evidence graphs from prior tasks. By constructing a retrieval-augmented model refinement framework, our proposed M-DESIGN dynamically weaves historical evidence to discover near-optimal modification paths. M-DESIGN features an adaptive retrieval mechanism that quickly calibrates the evolving transferability of edit-effect evidence from different sources. To handle out-of-distribution shifts, we introduce predictive task planners that extrapolate gains from multi-hop evidence, thereby reducing reliance on an exhaustive repository. Based on our model knowledge base of 67,760 graph neural networks across 22 datasets, extensive experiments demonstrate that M-DESIGN consistently outperforms baselines, achieving the search-space best performance in 26 out of 33 cases under a strict budget. Code and data are available at: https://anonymous.4open.science/r/M-DESIGN-245/.

Applications · Chemistry, Physics, and Earth Sciences

Shizheng Wen, Mingyuan chi, Tianwei Yu, Ben Moseley, Mike Yan Michelis, Pu Ren, Hao Sun, Siddhartha Mishra

We present a unified algorithmic framework for the numerical solution, constrained optimization, and physics-informed learning of PDEs with a variational structure. Our framework is based on a Galerkin discretization of the underlying variational forms, and its high efficiency stems from a novel highly-optimized and GPU-compliant TensorGalerkin framework for linear system assembly (stiffness matrices and load vectors). TensorGalerkin operates by tensorizing element-wise operations within a Python-level Map stage and then performs global reduction with a sparse matrix multiplication that performs message passing on the mesh-induced sparsity graph. It can be seamlessly employed downstream as i) a highly-efficient numerical PDEs solver, ii) an end-to-end differentiable framework for PDE-constrained optimization, and iii) a physics-informed operator learning algorithm for PDEs. With multiple benchmarks, including 2D and 3D elliptic, parabolic, and hyperbolic PDEs on unstructured meshes, we demonstrate that the proposed framework provides significant computational efficiency and accuracy gains over a variety of baselines in all the targeted downstream applications.

Optimization · Convex

Junchi Yang, Ziyang Zeng, Linxuan Pan, Murat Yildirim, Feng Qiu

In distributed machine learning, efficiently training across multiple agents with heterogeneous data distributions remains a central challenge. We address the problem of stochastic, strongly convex distributed optimization by applying accelerated gradient ascent to the dual variables and multi-step stochastic gradient descent (SGD) to the primal variables in the Lagrangian formulation. This approach naturally enables local computation, as the inner SGD loops require no inter-agent communication. We prove that the method converges for any number of local updates, attaining the optimal communication complexity when local computation is sufficient. Our analysis builds on an inexact accelerated gradient framework, where the partial gradient of the Lagrangian with respect to the dual variables is treated as an inexact gradient of the dual function. A notable byproduct of this framework is an algorithm that achieves optimal reproducibility guarantees under biased gradient estimates.

Theory · Everything Else

Cristiano Migali, Gianmarco Genalti, Alberto Maria Metelli, Marco Mussi

In domains such as recommender systems and information retrieval, learning from human-generated feedback is especially challenging because the information provided is often sparse and incomplete. In this work, we address the problem of learning the top-$k$ items from incomplete rankings. Most existing models for incomplete rankings rely on rigid assumptions regarding both the ranking model that generates the latent ranking and the censoring mechanism that determines which comparisons remain unobserved. On the one hand, the ranking model is often assumed to follow a Plackett-Luce (PL) or Mallows distribution. On the other hand, the censoring mechanism is typically assumed to be Missing Completely At Random (MCAR) or to exhibit well-behaved dependencies on the latent ranking, such as winner feedback or top-$h$ feedback. We introduce a new, general framework for learning from incomplete rankings that unifies and strictly generalizes the established frameworks in the literature. We consider the broad class of ranking models that satisfy the complete consensus property, which comprehends all widely adopted models, including PL and Mallows. Furthermore, we present a new preference-based feedback model, named positional censoring, which generalizes winner and top-$h$ feedback. We show that it is possible to learn in this general setting by presenting the PIRATE algorithm and providing a near-optimal instance-dependent bound to the sample complexity. Finally, we show that, under the PL ranking, PIRATE matches the sample complexity of state-of-the-art algorithms in the relevant scenarios of winner and top-$h$ feedback.

Deep Learning · Generative Models and Autoencoders

Yutian Zhao, Chao Du, Xiaosen Zheng, Tianyu Pang, Min Lin

Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs. Existing methods for diffusion models typically require access to model gradients or retraining, limiting their applicability in proprietary or large-scale settings. We propose a *nonparametric* attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images. Our approach is grounded in the analytical form of the optimal score function and naturally extends to multiscale representations, while remaining computationally efficient through convolution-based acceleration. In addition to producing spatially interpretable attributions, our framework uncovers patterns that reflect intrinsic relationships between training data and outputs, independent of any specific model. Experiments demonstrate that our method achieves strong attribution performance, closely matching gradient-based approaches and substantially outperforming existing nonparametric baselines.

Deep Learning · Generative Models and Autoencoders

Binhao Wang, Shihao Zhao, Bo Cheng, Qiuyu Ji, YuhangMa, Liebucha Wu, Shanyuan Liu, Dawei Leng, Yuhui Yin

Recent diffusion-based approaches have made substantial progress in image layer decomposition. However, accurately decomposing complex natural images remains challenging due to difficulties in occlusion completion, robust layer disentanglement, and precise foreground boundaries. Moreover, the scarcity of high-quality multi-layer natural image datasets limits advancement. To address these challenges, we propose **RevealLayer**, a diffusion-based framework that decomposes an RGB image into multiple RGBA layers, enabling precise layer separation and reliable recovery of occluded content in natural images. RevealLayer incorporates three key components: (1) a **Region-Aware Attention** module to disentangle hidden and visible layers; (2) an **Occlusion-Guided Adapter** to leverage contextual information to enhance overlapping regions; and (3) a **composite loss** to enforce sharp alpha boundaries and suppress residual artifacts. To support training and evaluation, we introduce **RevealLayer-100K**, a high-quality multi-layer natural image constructed through a collaboration between automated algorithms and human annotation, and further establish **RevealLayerBench** for benchmarking layer decomposition in general natural scenes. Extensive experiments demonstrate that RevealLayer consistently outperforms existing approaches in layer decomposition.

General Machine Learning · Online Learning, Active Learning and Bandits

Kellian Cottart, Theo Ballet, Djohan Bonnet, Damien Querlioz

Always-on edge systems must keep learning as conditions change under tight compute budgets and must detect unreliable predictions. Bayesian binary neural networks are attractive in this setting, but mean-field Bernoulli posteriors can saturate on long non-stationary streams, wiping out epistemic uncertainty and freezing plasticity. We propose BiMU, derived from a bounded-memory variational objective that balances stability, plasticity, and forgetting. BiMU combines a data term with controlled relaxation toward the prior and an uncertainty-dependent step size that prevents saturation and sustains informative uncertainty. This non-degenerate posterior enables fully online, buffer-free active querying via Monte Carlo disagreement, reducing label queries and backpropagation updates under imbalance. BiMU sustains learning and strong OOD detection on 1000-task Permuted-MNIST, and on OpenLORIS-Object achieves up to 32× label/update savings at matched accuracy under class imbalance and feature compression.