论文检索

Optimization · Large Scale, Parallel and Distributed

Andrej Jovanović, Alex Iacob, Mher Safaryan, Ionut-Vlad Modoranu, Lorenzo Sani, Shen, Xinchi Qiu, Dan Alistarh, Nicholas Lane

Distributed training of foundation models via $\texttt{DDP}$ is limited by interconnect bandwidth. While infrequent communication strategies reduce synchronization frequency, they remain bottlenecked by the memory and communication requirements of optimizer states. Low-rank optimizers can alleviate these constraints; however, in the local-update regime, workers lack access to the full-batch gradients required to compute low-rank projections, which degrades performance. We propose $\texttt{LoRDO}$, a principled framework unifying low-rank optimization with infrequent synchronization. We first demonstrate that, while global projections based on pseudo-gradients are theoretically superior, they permanently restrict the optimization trajectory to a low-rank subspace. To restore subspace exploration, we introduce a full-rank quasi-hyperbolic update. $\texttt{LoRDO}$ achieves near-parity with low-rank $\texttt{DDP}$ in language modeling and downstream tasks at model scales of $125$M--$720$M, while reducing communication by $\approx10\times$. Finally, we show that $\texttt{LoRDO}$ improves performance even more in very low-memory settings with small rank/batch size.

Applications · Computer Vision

Chengyu Zheng, Songlin Yang, Jin Huang, Honghua Chen, Weiming Wang, Haoran Xie, Fu Lee Wang, Mingqiang Wei

Point cloud registration can be categorized into rigid and non-rigid settings depending on the motion characteristics of the underlying objects. Rigid alignment assumes a single global transformation under which corresponding points remain geometrically consistent across scales, whereas non-rigid alignment involves spatially varying deformations, where geometric similarity holds only locally and semantic correspondence dominates at larger scales. This multi-scale discrepancy creates an optimization gap that has made unified registration particularly challenging. To this end, we propose RGGT, a Generative-Prior-Guided Transformer that unifies rigid and non-rigid registration within a shared optimization space. Through coordinated design at the representation, architecture, and supervision levels, RGGT jointly captures local geometric details and global structural semantics: generative priors enrich point features with unified geometric–semantic cues; a Global–Self–Cross Attention module models long-range structure, local interaction, and bidirectional cross-shape reasoning; and a dual correspondence–reconstruction objective provides consistent supervision for both deformation types. Extensive experiments on rigid (ModelNet40) and non-rigid (4DMatch) benchmarks demonstrate that RGGT achieves state-of-the-art accuracy across both rigid and non-rigid settings within a single unified framework.

Suyuan Liu, Shengfei Wei, Wenjing Yang, Shengju Yu, Siwei Wang, Xueqiong Li, Wenpeng Lu, Xinwang Liu

Existing fair multi-view clustering methods typically suffer from a severe trade-off between clustering utility and fairness, while incurring prohibitive quadratic complexity on large-scale datasets. To address these challenges, we propose Causal Disentangled Anchor Learning (CDAL), a novel framework that achieves scalable fairness via structural disentanglement. Guided by a structural causal model perspective, CDAL utilizes a dual-anchor mechanism to structurally separate latent representations into orthogonal semantic and sensitive subspaces. We further ensure statistical independence through a linearized Hilbert-Schmidt Independence Criterion (HSIC) constraint, which is optimized via an efficient alternating scheme. Theoretically, we prove the identifiability of the disentangled factors and guarantee the algorithm's global convergence and linear time complexity $\mathcal{O}(n)$. Extensive experiments on large-scale benchmarks demonstrate that CDAL outperforms state-of-the-art methods, achieving a superior utility-fairness trade-off.

Deep Learning · Generative Models and Autoencoders

Jinbin Bai, Yu Lei, Qingyu Shi, Aosong Feng, Yi Xin, Zhuoran Zhao, Fei Shen, Kaidong Yu, Xiangtai Li

Aligning large visual generative models with human feedback is often performed through pairwise preference optimization. While such approaches are conceptually simple, they fundamentally rely on annotated pairs, limiting scalability in settings where feedback is collected as independent scalar ratings. In this work, we revisit the KL-regularized alignment objective and show that the optimal policy implicitly compares each sample’s reward to an instance-specific baseline that is generally intractable. We propose a threshold-guided alignment framework that replaces this oracle baseline with a data-driven global threshold estimated from empirical score statistics. This formulation turns alignment into a binary decision task on unpaired data, enabling effective optimization directly from scalar feedback. We further incorporate a confidence weighting term to emphasize samples whose scores deviate strongly from the threshold, improving sample efficiency. Experiments across both diffusion and masked generative paradigms, spanning three test sets and five reward models, show that our method consistently improves preference alignment over previous methods. These results position our threshold-guided framework as a simple yet principled alternative for aligning visual generative models without paired comparisons.

Applications · Chemistry, Physics, and Earth Sciences

Chenghao Jia, Mengdi Liu, Hong Chang, Shiguang Shan, Xilin Chen

Elucidating molecular structures from spectra is a foundational problem in chemical and materials characterization, yet remains challenging due to spectral ambiguity and the vast molecular space. Although recent diffusion-based generators show strong promise for spectra-conditioned elucidation, existing methods struggle to learn robust spectra-structure relationships from limited paired data when relying solely on global spectral representation. Moreover, the repeated full sampling inference strategy incurs substantial computation overhead. To address these limitations, we propose MAST, a Motif-Augmented diffusion framework with Search Tree, for joint 2D-3D spectroscopic molecular structure elucidation. MAST introduces explicit, interpretable motif priors as intermediate evidences throughout denoising, reducing conditional ambiguity and facilitating spectra-conditioned optimization. We further cast diffusion sampling as reward-guided tree search to prioritize high-reward denoising trajectories, yielding a compact set of spectra-consistent candidates under limited budgets. On the QM9S multi-spectra benchmark, MAST achieves 94.89% exact recovery and improves 3D fidelity, while preserving high chemical validity and stability.

Deep Learning · Large Language Models

Niccolò Avogaro, Nayanika Debnath, Li Mi, Thomas Frick, Junling Wang, Zexue He, Hang Hua, Konrad Schindler, Mattia Rigotti

Despite recent successes, *test-time scaling* $-$i.e., dynamically expanding the token budget during inference as needed$-$ remains brittle for vision-language models (VLMs): unstructured chains-of-thought about images entangle perception and reasoning, leading to long, disorganized contexts where small perceptual mistakes may cascade into completely wrong answers. Moreover, expensive reinforcement learning with hand-crafted rewards is required to achieve good performance. Here, we introduce SPARC (Separating Perception And Reasoning Circuits), a modular framework that explicitly decouples visual perception from reasoning. Inspired by sequential sensory-to-cognitive processing in the brain, SPARC implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer. This separation enables independent test-time scaling with asymmetric compute allocation (e.g., prioritizing perceptual processing under distribution shift), supports selective optimization (e.g., improving the perceptual stage alone when it is the bottleneck for end-to-end performance), and accommodates compressed contexts by running global search at lower image resolutions and allocating high-resolution processing only to selected regions, thereby reducing total visual tokens count and compute. Across challenging visual reasoning benchmarks, SPARC outperforms monolithic baselines and strong visual-grounding approaches. For instance, SPARC improves the accuracy of Qwen3VL-4B on the $V^*$ VQA benchmark by 6.7 percentage points, and it surpasses "thinking with images" by 4.6 points on a challenging OOD task despite requiring a 200$\times$ lower token budget.

Applications · Computer Vision

Lei Li, Angela Dai

We present HOI-PAGE, a new approach that prioritizes part-level affordance reasoning to generate high-fidelity 4D human-object interactions (HOIs) from text prompts in a zero-shot fashion. In contrast to prior works that focus on global, whole body-object motion synthesis, our approach explicitly reasons about the underlying fine-grained mechanics of interactions using large language models (LLMs). We capture this reasoning in a structured part affordance graph (PAG) representation, serving as a high-level interaction scaffolding to guide a three-stage synthesis: first, decomposing input 3D objects into semantic parts; then, generating reference HOI videos from text prompts to extract part-based motion constraints; and finally, optimizing for 4D HOI motion sequences that mimic the reference dynamics while satisfying part-level contact constraints. Extensive experiments show that our approach is flexible and capable of generating complex multi-object or multi-person interaction sequences, with significantly improved realism and text alignment for zero-shot 4D HOI generation.

Applications · Computer Vision

Zhifeng Gu, Yuqi Wang, Bing WANG

Relative spatial relations provide a compact description of spatial structure, serving as a key component of relative spatial reasoning in 3D layout generation. Recent works leverage Multimodal Large Language Models (MLLMs) to infer these relations, but the inferred relations are often unreliable and are resolved by post-hoc heuristics at the cost of semantic fidelity. In this paper, we propose R$^3$L, a general framework that improves the reliability and consistency of relative spatial reasoning for 3D layout generation. Our key motivation is that multi-hop reasoning requires repeated reference-frame shifts, which accumulate errors and lead to semantic and metric drift. To mitigate this, we propose invariant spatial decomposition to shorten relations chains, and consistent spatial imagination that uses an imagine-and-revise loop to encourage self-consistency during relation inference. We further design supportive spatial optimization that eases pose optimization by global-to-local coordinate re-parameterization. Extensive experiments across diverse scene types and instructions demonstrate that R$^3$L improves layout feasibility and semantic consistency. Notably, our analysis shows that resolving frame-induced inconsistencies during reasoning is crucial for reliable multi-hop relative spatial reasoning. Code will be released upon acceptance.

Social Aspects · Accountability, Transparency, and Interpretability

Viet-Hung Tran, Zichi Zhang, Ngoc Doan, Xuan Hoang Nguyen, Phi Nguyen, Yimeng An, Peixin Li, Hans Vandierendonck, Ira Assent, Thai Son Mai

Deep Neural Networks (DNNs) achieve outstanding performance in Time Series Classification (TSC) yet remain opaque "black boxes", hindering their adoption in sensitive domains. While Explainable AI (XAI) aims to bridge this gap, existing TSC XAI methods rely on a single perspective and incur significant computational costs, making them difficult to integrate into real-time applications. To overcome these challenges, we propose a framework, containing two key techniques called Instance-level Multi-Expert Knowledge Distillation (IMEKD) and Semi-Amortized Optimization Explanation (SAOE). Unlike static methods, our IMEKD approach bridges the gap between explanation methods by dynamically selecting the best attribution map from a pool of "XAI experts" for each instance. We then distill this instance-optimal knowledge into a DNN using our SAOE framework, a dual-stage process that learns a global attribution via distillation and refines it using faithfulness and robustness losses, aligning the optimization with objectives of human evaluation. To the best of our knowledge, this is the first work to unify multi-expert selection with semi-amortized optimization for TSC XAI. Also, we introduce a Faithfulness-Preserving Segmentation (FPS) mechanism that converts point-wise maps into interpretable segments without sacrificing fidelity to align explanations with human intuition. Comprehensive experiments on four synthetic datasets, a ECG dataset with human-verified ground truth, and 11 multivariate UEA benchmarks across three DNN architectures show that our framework significantly outperforms the current state-of-the-art (SOTA) in terms of faithfulness, robustness, and computational efficiency.

Reinforcement Learning · Online

Chenyi Li, Yuan Zhang, Bo Wang, Guoqing Ma, Wei Tang, Haoyang Huang, Nan Duan

Reinforcement learning with verifiable rewards has shown notable effectiveness in enhancing large language models (LLMs) reasoning performance, especially in mathematics tasks. However, such improvements often come with reduced outcome diversity, where the model concentrates probability mass on a narrow set of solutions. Motivated by diminishing-returns principles, we introduce a set level diversity objective defined over sampled trajectories using kernelized similarity. Our approach derives a leave-one-out marginal contribution for each sampled trajectory and integrates this objective as a plug-in advantage shaping term for policy optimization. We further investigate the contribution of a single trajectory to language model diversity within a distribution perturbation framework. This analysis theoretically confirms a monotonicity property, proving that rarer trajectories yield consistently higher marginal contributions to the global diversity. Extensive experiments across a range of model scales demonstrate the effectiveness of our proposed algorithm, consistently outperforming strong baselines in both Pass@1 and Pass@K across various benchmarks.

Deep Learning · Foundation Models

Keondo Park, Younghoon Na, Yourim Choi, Hyunwoo Ryu, Hyun-Woo Shin, Hyung-Sin Kim

While the shift toward unified foundation models has revolutionized many deep learning domains, sleep medicine remains largely restricted to task-specific models that focus on localized micro-structure features. These approaches often neglect the rich, multi-modal context of Polysomnography (PSG) and fail to capture the global macro-structure of a full night's sleep. To address this, we introduce SleepMaMi, a Sleep Foundation Model engineered to master both hour-long sleep architectures and fine-grained signal morphologies. Our framework utilizes a hierarchical dual-encoder design: a Macro-Encoder to model full-night temporal dependencies and a Micro-Encoder to capture short-term characteristics from biosignals. Macro-Encoder is trained via Demographic-Guided Contrastive Learning, which aligns overnight sleep patterns with objective subject metadata, such as age and sex, to refine global representations. Micro-Encoder is optimized via a hybrid Masked Autoencoder (MAE) and multi-modal contrastive objective. Pre-trained on a massive corpus of >20,000 PSG recordings (15.8K hours), SleepMaMi outperforms existing foundation models across a diverse suite of downstream tasks, demonstrating superior generalizability and label-efficient adaptation for clinical sleep analysis.

Applications · Neuroscience, Cognitive Science

Jea Kwon, Dong-Kyum Kim, Jiwon Kim, Yonghyun Kim, Woong Kook, MEEYOUNG CHA

Memory formation is fundamental to intelligence, yet whether deep neural networks preserve identifiable memory traces—analogous to biological memory units—remains an open question. This work introduces a geometric framework to identify such "AI engrams," by formalizing the neuroscientific criteria of specificity, reactivation, sufficiency, and necessity into a constrained inverse problem. We derive a closed-form estimator that isolates individual memory traces from globally entangled parameters. Theoretical analysis reveals that this biologically-derived solution corresponds to a natural gradient update on the parameter manifold. AI engrams enable surgical manipulation of learned knowledge: any subset of memories can be composed or erased through linear arithmetic, without iterative optimization. Experiments ranging from simple MLPs to LLMs demonstrate the causal validity and substantial scalability of AI engrams. Together, these results bridge theories of biological memory and artificial representation learning, offering geometric insight into how deep networks simultaneously support functional specificity within distributed storage.

Applications · Neuroscience, Cognitive Science

Jea Kwon, Dong-Kyum Kim, Jiwon Kim, Yonghyun Kim, Woong Kook, MEEYOUNG CHA

Memory formation is fundamental to intelligence, yet whether deep neural networks preserve identifiable memory traces—analogous to biological memory units—remains an open question. This work introduces a geometric framework to identify such "AI engrams," by formalizing the neuroscientific criteria of specificity, reactivation, sufficiency, and necessity into a constrained inverse problem. We derive a closed-form estimator that isolates individual memory traces from globally entangled parameters. Theoretical analysis reveals that this biologically-derived solution corresponds to a natural gradient update on the parameter manifold. AI engrams enable surgical manipulation of learned knowledge: any subset of memories can be composed or erased through linear arithmetic, without iterative optimization. Experiments ranging from simple MLPs to LLMs demonstrate the causal validity and substantial scalability of AI engrams. Together, these results bridge theories of biological memory and artificial representation learning, offering geometric insight into how deep networks simultaneously support functional specificity within distributed storage.

Optimization · Zero-order and Black-box Optimization

Geng Li, Yuxin Peng

While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior-free scanning, or depend on static prior-driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose **BVS** (**B**ayesian **V**isual **S**earch), a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub-linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.

Optimization · Stochastic

Haoxuan Wang, Xinchen Du, Sen Na

Reliable decision-making with streaming data requires principled uncertainty quantification of online methods. While first-order methods enable efficient iterate updates, their inference procedures still require updating proper (covariance) matrices, incurring $O(d^2)$ time and memory complexity, and are sensitive to ill-conditioning and noise heterogeneity of the problem. This costly inference task offers an opportunity for more robust second-order methods, which are, however, bottlenecked by solving Newton systems with $O(d^3)$ complexity. In this paper, we address this gap by studying an online Newton method with Hessian averaging, where the Newton direction at each step is approximately computed using a *sketch-and-project solver with Nesterov's acceleration*, matching $O(d^2)$ complexity of first-order methods. For the proposed method, we quantify its uncertainty arising from both random data and randomized computation. Under standard smoothness and moment conditions, we establish global almost-sure convergence, prove asymptotic normality of the last iterate with a limiting covariance characterized by a Lyapunov equation, and develop a fully online covariance estimator with non-asymptotic convergence guarantees. We also connect the resulting uncertainty quantification to that of exact and sketched Newton methods without Nesterov's acceleration. Extensive experiments on regression models demonstrate the superiority of the proposed method for online inference.

Social Aspects · Robustness

Lucas D. Konrad, Nikolas Kuschnig

Identifying *most influential sets* (MIS) – size-$k$ subsets whose removal maximally changes a target estimand – is typically infeasible because it requires searching over $\binom{n}{k}$ subsets. We show that, for a broad class of estimands whose leave-set-out effect admits a linear-fractional form, the MIS problem reduces to a one-parameter sequence of top-$k$ selections. Using Dinkelbach's method, we obtain an efficient algorithm that runs in $O(n)$ per iteration and terminates in finitely many steps. We show that our approach returns globally optimal sets for univariate settings, such as average treatment effect estimation in randomized experiments. For partial linear models, we establish selection consistency under Neyman orthogonality and mild first-stage stability. We validate our method through simulations and real-world applications – recovering MIS that were previously computationally inaccessible.

Theory · Active Learning and Interactive Learning

Vincent Cohen-Addad, Sasidhar Kunapuli, Vahab Mirrokni, Mahdi Nikdan, David Woodruff, Samson Zhou

In the data selection problem, the objective is to choose a small, representative subset of data that can be used to efficiently train a machine learning model. Sener and Savarese [ICLR 2018] showed that, given an embedding representation of the data and suitable geometric assumptions, heuristics based on $k$-center clustering can be used to perform data selection. This perspective was further explored by Axiotis et. al. [ICML 2024], who proposed a data selection approach based on $k$-means clustering and sensitivity sampling. However, these methods rely on the assumption that the dataset exhibits intrinsic geometric structure that can be effectively captured by clustering, whereas many modern datasets instead possess global algebraic structure that is better exploited by low-rank approximation or principal component analysis. In this paper, we introduce a new data selection framework based on low-rank approximation and residual-based sampling, formulated through the lens of row subset selection and loss-preserving coreset construction. Given an embedding representation of the data satisfying mild regularity conditions, which can be interpreted as algebraic or angular notions of Lipschitz continuity, we show that it is possible to select a weighted subset of $\tilde{O}\left(k + \frac{1}{\varepsilon^2}\right)$ data points whose average loss approximates the average loss over the full dataset within a $(1+\varepsilon)$ relative error, up to an additive $\varepsilon \Phi_k$ term, where $\Phi_k$ denotes the optimal rank-$k$ approximation cost of the embedding matrix. We complement these theoretical guarantees with empirical evaluations, demonstrating that on a range of real-world datasets, our data selection approach achieves improved performance over prior strategies based on uniform sampling or clustering-based sensitivity sampling.

General Machine Learning · Clustering

Yiming Wang, Qun Li, Dongxia Chang, Jie Wen, Hua Dai, Fu Xiao

Deep Multi-View Clustering (MVC) aims to extract a unified semantic consensus from diverse data sources without supervision. However, current approaches relying on flat Euclidean embeddings often fail to model data uncertainty, resulting in rigid alignment where high-quality views are forced to drift toward corrupted ones. To address these challenges, we propose the Hyperbolic Asymmetric Multi-view Clustering (HAMC) framework. By embedding features into the Poincaré ball model, HAMC leverages the exponential volume growth of hyperbolic geometry to optimize cluster separability. It pushes high-confidence representations toward the boundary while retaining noisy ones near the origin. To mitigate noise, we introduce an asymmetric view alignment mechanism, enabling reliable views to unidirectionally guide unreliable ones. Furthermore, a consensus-aware cluster learning strategy is designed to construct robust global pseudo-labels via a confidence-based screening scheme, refining the cluster structure. Extensive experiments against 13 baselines demonstrate that HAMC achieves state-of-the-art performance.

Applications · Robotics

Rufeng Chen, Yue Chang, Xiaqiang Tang, Hechang Chen, Sihong Xie

Open-vocabulary navigation requires embodied agents to manage significant perception uncertainty stemming from semantic ambiguity and model errors. However, most existing works settle for local optimal deterministic approaches, depriving complex navigation decision-making over multiple composite possibilities that are critical for globally better solutions. In this paper, we propose Probabilistic Scene Graph Navigation (PSG-Nav), which constructs a 3D Probabilistic Scene Graph that uses full semantic categorical distributions to account for perception uncertainty. To efficiently use the local distributions to compose and reason about the optimal navigation landmarks, we propose Multiverse Decision to sample multiple most likely world settings from the joint distribution, and evaluate navigation landmarks based on the compatibility between landmarks and multiverses. To mitigate false positives due to epistemic uncertainty in open-vocabulary navigation, we introduce the Evidential Experience Calibrator, which enables online lifelong adaptation by cross-validating detections against memories of past successes and failures. Extensive experiments on widely-used benchmarks MP3D, HD3D, and HSSD demonstrate that PSG-Nav establishes new state-of-the-art results, achieving a Success Rate of 66.1%, 44.8%, and 67.9%, respectively.

Deep Learning · Other Representation Learning

Yeqin Zhang, Yunfei Wang, Jiaxuan Chen, Ke Qin, Yizheng Zhao, Cam-Tu Nguyen

Sentence representations are foundational to many Natural Language Processing (NLP) applications. While recent methods leverage Large Language Models (LLMs) to derive sentence representations, most rely on final-layer hidden states, which are optimized for next-token prediction and thus often fail to capture global, sentence-level semantics. This paper introduces a novel perspective, demonstrating that attention value vectors capture sentence semantics more effectively than hidden states. We propose *Value Aggregation (VA)*, a simple method that pools token values across multiple layers and token indices. In a training-free setting, VA outperforms other LLM-based embeddings, even matches or surpasses the ensemble-based MetaEOL. Furthermore, we demonstrate that when paired with suitable prompts, the layer attention outputs can be interpreted as aligned weighted value vectors. Specifically, the attention scores of the last token function as the weights, while the output projection matrix ($W_O$) aligns these weighted value vectors with the common space of the LLM residual stream. This refined method, termed *Aligned Weighted VA (AlignedWVA)*, achieves state-of-the-art performance among training-free LLM-based embeddings, outperforming the high-cost MetaEOL by a substantial margin. Finally, we highlight the potential of obtaining strong LLM embedding models through fine-tuning Value Aggregation.