论文检索

Applications · Time Series

guanqun zhao, Yitong Liu, Jiaxuan Fang, Hongwen Yang, Yufei Mao

Standard Self-Supervised Learning (SSL) for Automatic Modulation Recognition (AMR) struggles with ineffective isotropic augmentations, spectral instability, and semantic drift. To address these challenges, we propose Dynamic-Consistency Contrastive Learning (DyCo-CL), a geometry-aware framework that couples Virtual Adversarial Augmentation (VAA) with a semantic consistency loss. We provide a theoretical analysis indicating that this strategy acts as an implicit spectral regularizer for the encoder, enabling stable manifold exploration. Complementing this, our Signal-Adaptive Swin Backbone with fixed-window attention improves structural stability by constraining attention locality, while a Hybrid Knowledge Fusion module anchors representations with physical priors. Experiments on RML benchmarks show that DyCo-CL achieves a 6.27% accuracy gain in 1-shot settings over prior methods.

Deep Learning · Sequential Models, Time series

Ruichao Guo, Xingyao Han, Wenshui Luo, Zhe Liu, Chen Gong, Hesheng Wang

Point forecasting for graph-structured multivariate time series is a fundamental problem, but rigorous uncertainty quantification for such predictions is still underexplored. Conformal prediction (CP) offers uncertainty estimation with a solid coverage guarantee under the exchangeability assumption, which requires the joint data distribution to be unchanged under permutation. However, in graph-structured time series, inherent cross-node coupling can violate the exchangeability condition, making direct application of CP unreliable. Inspired by the spectral graph theory, such coupling resides in global trends and can be characterized by the low-frequency components, while high-frequency components are nearly exchangeable. Therefore, we propose a novel concept named **S**pectral **G**raph **C**onditional **E**xchangeability (SGCE), which conditions exchangeable high-frequency components on low-frequency ones to preserve global trends and enable effective CP in the spectral domain. Based on SGCE, we further propose **S**pectral **C**onformal prediction via w**A**ve**LE**t transform (SCALE). SCALE uses graph wavelets to decompose low/high-frequency components and conformalizes high-frequency residuals via adaptive gating over a low-frequency embedding. Experimental results on real-world traffic datasets show that SCALE not only achieves valid coverage but also consistently improves the coverage-efficiency trade-off over the state-of-the-art CP methods.

Theory · Domain Adaptation and Transfer Learning

Zeyu Ma, Jiaqi Huang, Yitong Qin, Ziqiang Zheng, Jiwei Wei, Jie Zou, Yang Yang, Heng Tao Shen

Domain Adaptive Object Detection (DAOD) addresses the challenge of transferring object detectors from labeled source domains to unlabeled target domains. Existing domain adaptation methods primarily rely on feature distribution alignment, which enhances domain-invariant features (statistical invariance) but also inadvertently increases inherent domain-common spurious factors (e.g., common environmental contexts), which act as shortcut features rather than the true causal factors for object classification. We propose Dynamic Causal Refinement (DCR), a novel framework that establishes a closed-loop feedback mechanism between data augmentation and model optimization to progressively refine causal features. Specifically, we design Semantic Prediction Consistency (SPC) to filter domain-specific spurious factors and establish a robust statistical invariance, and Discrepancy-Guided Causal Refinement (DGCR) to actively suppress the dependence on domain-common spurious factors via spectral perturbation for causal refinement. This process encourages the detector to suppress its reliance on shortcut features and instead prioritize semantically meaningful causal representations. Extensive experiments on standard benchmarks demonstrate that our method outperforms state-of-the-art counterparts significantly.

Optimization · Stochastic

Zitao Song, Cedar Site Bai, Zhe Zhang, Brian Bullins, David Gleich

Adaptive methods like Adam have become the *de facto* standard for large-scale vector and Euclidean optimization due to their coordinate-wise adaptation with a second-order nature. More recently, matrix-based spectral optimizers like Muon (Jordan et al., 2024b) show the power of treating weight matrices as matrices rather than long vectors. Linking these is hard because many natural generalizations are not feasible to implement, and we also cannot simply move the Adam adaptation to the matrix spectrum. To address this, we reformulate the AdaGrad update and decompose it into a variance adaptation term and a scale-invariant term. This decoupling produces **DeVA** (**De**coupled **V**ariance **A**daptation), a framework that bridges between vector-based variance adaptation and matrix spectral optimization, enabling a seamless transition from Adam to adaptive spectral descent. Extensive experiments across language modeling and image classification demonstrate that DeVA consistently outperforms state-of-the-art methods such as Muon and SOAP (Vyas et al., 2024), reducing token usage by around 6.6\%. Theoretically, we show that the variance adaptation term effectively improves the blockwise smoothness, facilitating faster convergence.

Applications · Chemistry, Physics, and Earth Sciences

Chenghao Jia, Mengdi Liu, Hong Chang, Shiguang Shan, Xilin Chen

Elucidating molecular structures from spectra is a foundational problem in chemical and materials characterization, yet remains challenging due to spectral ambiguity and the vast molecular space. Although recent diffusion-based generators show strong promise for spectra-conditioned elucidation, existing methods struggle to learn robust spectra-structure relationships from limited paired data when relying solely on global spectral representation. Moreover, the repeated full sampling inference strategy incurs substantial computation overhead. To address these limitations, we propose MAST, a Motif-Augmented diffusion framework with Search Tree, for joint 2D-3D spectroscopic molecular structure elucidation. MAST introduces explicit, interpretable motif priors as intermediate evidences throughout denoising, reducing conditional ambiguity and facilitating spectra-conditioned optimization. We further cast diffusion sampling as reward-guided tree search to prioritize high-reward denoising trajectories, yielding a compact set of spectra-consistent candidates under limited budgets. On the QM9S multi-spectra benchmark, MAST achieves 94.89% exact recovery and improves 3D fidelity, while preserving high chemical validity and stability.

Deep Learning · Robustness

Alireza Abdollahpourrostam, Nikolaos Dimitriadis, Adam Hazimeh, Pascal Frossard

Fine-tuning large pre-trained models on a target distribution often improves in-distribution (ID) accuracy, but at the cost of out-of-distribution (OOD) robustness as representations specialize to the fine-tuning data. Weight-space ensembling methods, such as Model Soups, mitigate this effect by averaging multiple checkpoints, but they are computationally prohibitive, requiring the training and storage of dozens of fine-tuned models. In this paper, we introduce MonoSoup, a simple, data-free, hyperparameter-free, post-hoc method that achieves a strong ID–OOD balance using _only a single_ checkpoint. Our method applies Singular Value Decomposition (SVD) to each layer’s update and decomposes it into high-energy directions that capture task-specific adaptation and low-energy directions that introduce noise but may still encode residual signals useful for robustness. MonoSoup then uses entropy-based effective rank to automatically re-weigh these components with layer-wise coefficients that account for the spectral and geometric structure of the model. Experiments on CLIP models fine-tuned on ImageNet and evaluated under natural distribution shifts, as well as on Qwen language models tested on mathematical reasoning and multiple-choice benchmarks, show that this plug-and-play approach is a practical and effective alternative to multi-checkpoint methods, retaining much of their benefits without their computational overhead.

Probabilistic Methods · Monte Carlo and Sampling Methods

Yunbum Kook, Shunshi Zhang

We investigate the complexity of covariance matrix estimation for Gibbs distributions based on dependent samples from a Markov chain. We show that when $π$ satisfies a Poincaré inequality and the chain possesses a spectral gap, we can achieve similar sample complexity using MCMC as compared to an estimator constructed using i.i.d. samples, with potentially much better query complexity. As an application of our methods, we show improvements for the query complexity in both constrained and unconstrained settings for concrete instances of MCMC. In particular, we provide guarantees regarding isotropic rounding procedures for sampling uniformly on convex bodies.

Applications · Robotics

Shuxin Cao, Liquan Wang, Walker Byrnes, Yiye Chen, Yilun Du, Animesh Garg

In this paper, we identify a semantic decomposition in robot action sequences, separating task-level motion intent from execution-level refinements. By analyzing actions in the spectral domain using the discrete cosine transform (DCT), we observe that low-frequency components capture global motion trajectories, while high-frequency components encode precise timing, alignment, and contact behaviors. Motivated by this structure, we propose Causal Spectral Policy (CSP), which models action generation as a causal coarse-to-fine process: coarse motion is predicted from observation and language, and fine corrections are generated conditionally on the realized trajectory. Across simulation and real-world evaluations, CSP consistently outperforms strong baselines on precision-sensitive manipulation tasks. Additionally, we propose human-inspired teleoperation noise injection as a data augmentation method under which our approach demonstrates strong robustness to noisy demonstrations.

Probabilistic Methods · Monte Carlo and Sampling Methods

Lorenzo Baldassari, Josselin Garnier, Knut Solna, Maarten de Hoop

Designing algorithms that can explore multimodal target distributions accurately across successive refinements of an underlying high-dimensional problem is a central challenge in sampling. Annealed Langevin dynamics (ALD) is a widely used alternative to classical Langevin since it often yields much faster mixing on multimodal targets, but there is still a gap between this empirical success and existing theory: when, and under which design choices, can ALD be guaranteed to remain stable as dimension increases? In this paper, we help bridge this gap by providing a uniform-in-dimension analysis of continuous-time ALD for multimodal targets that can be well-approximated by Gaussian mixture models. Along an explicit annealing path obtained by progressively removing Gaussian smoothing of the target, we identify sufficient spectral conditions—linking smoothing covariance and the covariances of the Gaussian components of the mixture—under which ALD achieves a prescribed accuracy within a single, dimension-uniform time horizon. We then establish dimension-robustness to imperfect initialization and score approximation: under a misspecified-mixture score model, we derive explicit conditions showing that preconditioning the ALD algorithm with a sufficiently decaying spectrum is necessary to prevent error terms from accumulating across coordinates and destroying dimension-uniform control. Finally, numerical experiments illustrate and validate the theory.

Deep Learning · Generative Models and Autoencoders

Zhiyuan Ouyang, Simei Huang, Zuokai Wen, Xiangyun Zhang, Junchi Yan

A probability distribution $\mathbb{P}$ is a measure on a $\sigma$-algebra, assigning mass to sets rather than points. This poses a challenge for the training of neural networks, which often struggle to reconstruct the global topology of continuous manifolds from sparse samples. We mitigate this issue by innovatively transforming $\mathbb{P}$ into a continuous function via spectral methods, providing theoretical guarantees for the convergence of the learned distribution to the true distribution. Specifically, we introduce a network, PriSpecNet, with a single-function evaluation (1-NFE) Pseudoinverse Lévy Inversion (PiLI) solver that regards generation as a fast analytical problem, eliminating the need for iterative numerical integration. By reformulating the generation in the spectral domain, we bypass the computationally expensive sampling trajectory while maintaining full compatibility with the stochastic interpolants. We test our PriSpecNet in two applications. For time series, it unifies generation and forecasting, outperforming state-of-the-art (SOTA) baselines with Context-FID reductions of 50.0\%, 41.5\%, 80.6\%, and 63.1\% on Sines, Solar, ETTh, and Stock benchmarks, respectively, also decreasing forecasting MSE by 29.8\% on Solar and 23.8\% on Stock. For vision on ImageNet $256 \times 256$, 1-NFE PiLI achieves a competitive FID of **1.66** using only **26** Gflops, representing a **170** $\times$ reduction in total Gflops compared to the 4,436 Gflops required by the advanced 25-NFE DPM-Solver++.

Probabilistic Methods · Spectral Methods

Genki Osada

While diffusion models enable new approaches for estimating Local Intrinsic Dimension (LID), existing methods fail in high-dimensional spaces where noise from vast normal directions overwhelms the tangent signal. We propose Local Hessian Spectral Dimension (LHSD), which resolves this by applying spectral filtering to the log-density Hessian, explicitly cutting off large eigenvalues associated with normal directions to count zero-curvature tangent directions. Implemented using Stochastic Lanczos Quadrature (SLQ), LHSD avoids full Hessian construction, achieving linear scalability with dimension $D$. Experiments on synthetic and real data confirm LHSD’s superior robustness and its utility in detecting memorization in large-scale diffusion models.

Theory · Online Learning and Bandits

Sifan Yang, Wenhao Yang, Wei Jiang, Lijun Zhang

We investigate decentralized online convex optimization with compressed communication, where $n$ learners connected by a network collaboratively minimize a sequence of global loss functions using only local information and compressed data from neighbors. Prior work has established regret bounds of $O(\max\\{\omega^{-2}\rho^{-4}n^{1/2},\omega^{-4}\rho^{-8}\\}n\sqrt{T})$ and $O(\max\\{\omega^{-2}\rho^{-4}n^{1/2},\omega^{-4}\rho^{-8}\\}n\ln{T})$ for convex and strongly convex functions, respectively, where $\omega\in(0,1]$ is the compression quality factor and $\rho<1$ is the spectral gap of the communication matrix. However, these regret bounds suffer from a prohibitively high quadratic or even quartic dependence on $\omega^{-1}$. Moreover, the super-linear dependence on $n$ is also undesirable. To overcome these shortcomings, we propose a novel algorithm that achieves improved regret bounds of $\tilde{O}(\omega^{-1/2}\rho^{-1}n\sqrt{T})$ and $\tilde{O}(\omega^{-1}\rho^{-2}n\ln{T})$ for convex and strongly convex functions, respectively. The primary idea is to design a two-level blocking update framework incorporating two novel ingredients: an online gossip strategy and an error compensation scheme, which work together to promote a better consensus among learners. Furthermore, we establish the first lower bounds for this problem, justifying the optimality of our results with respect to both $\omega$ and $T$. Additionally, we consider the bandit feedback scenario and extend our method with the classical gradient estimators to enhance existing regret bounds.

Deep Learning · Large Language Models

Wentao Hu, Zeyu Zhu, Mingkuan Zhao, Zhenhua An, Yanbo Zhai, Shanhong yu, Huilin Zhou, Xin Lai, Xiaoyan Zhu, Jiayin Wang

Sparse Mixture-of-Experts (MoE) architectures scale model capacity efficiently but suffer from massive static parameter footprints, creating significant deployment burdens on memory-constrained hardware. Existing post-training pruning methods often rely on scalar statistics, ignoring the representational geometry of expert feature spaces. This leads to sub-optimal resource allocation across layers and the retention of redundant experts. To address this, we propose a Rank-aware Geometric Expert Pruning (RaGEP) framework to compress MoE models by analyzing the geometric properties of expert activations. First, in the inter-layer allocation stage, we introduce a Rank-aware budget allocation mechanism that adaptively assigns expert budgets based on the effective rank of layer-wise representations. Second, in the intra-layer selection stage, we propose a Spectral-Salience Pruning metric that harmonizes subspace orthogonality and activation magnitude to identify high-energy orthogonal experts. Extensive experiments across MoE models of different scales show that our method consistently outperforms state-of-the-art baselines on a diverse set of zero-shot tasks, while reducing model size and inference cost. Code is available at supplementary material.

Deep Learning · Generative Models and Autoencoders

Bowen Xue, Giuseppe Guarnera, Shuang Zhao, Zahra Montazeri

Current video diffusion models generate visually compelling content but often violate basic laws of physics, producing subtle artifacts like rubber-sheet deformations and inconsistent object motion. We introduce a frequency-domain physics prior that improves motion plausibility without modifying model architectures. Our method decomposes common rigid motions (translation, rotation, scaling) into lightweight spectral losses computed on a low-frequency subset. Applied to Open-Sora, MVDIT, and Hunyuan, our approach improves both motion accuracy and action recognition by ~11\% on average on OpenVID-1M (relative), while maintaining visual quality. User studies show 74--83\% preference for our physics-enhanced videos. It also reduces warping error by 22--37\% (depending on the backbone) and improves temporal consistency scores. These results indicate that simple, global spectral cues are an effective drop-in regularizer for physically plausible motion in video diffusion.

Deep Learning · Generative Models and Autoencoders

Yingyan Hou, Xianchi Dong, Chao Ren, Wanxuan Lu, Zihan Wei, Hongfeng Yu, Yixiao Wang, Yaning Zhou

Object insertion has emerged as a promising augmentation paradigm to solve the label scarcity and long-tail distributions in remote sensing. It aims to generate training samples by synthesizing target instances onto real backgrounds. However, existing methods have three critical issues: (i) Semantic placement inconsistency, (ii) Radiometric inconsistency with illumination and atmospheric conditions, and (iii) Textural discontinuity. To cope with these issues, we propose a physics-aware method, called "Plan, Decouple, Assimilate" (PDA), for generating high-fidelity training samples. In the planning stage, the Planning (P) module automatically generates geometrically bounding boxes. In the generation stage, we design a dual-module model to generate the target instance: the Decoupling (D) module employs Asymmetric Spectral Adaptation Decoupling to disentangle structural identity from environmental illumination, while the Assimilation (A) module utilizes Neighborhood-Aware Texture Assimilation to harmonize the local manifold. By strategically integrating these modules, PDA enforces multi-level consistency spanning global geometry to local micro-textures. Extensive experiments verify that PDA consistently outperforms existing state-of-the-art methods in generative quality, reducing whole-image FID by 15.7%, and substantially improves downstream detection performance, boosting average mAP50 by 15.9% over the real-data baseline.

Applications · Chemistry, Physics, and Earth Sciences

Dongzhe Zheng, Tao Zhong, Christine Allen-Blanchette

In this paper, we study solution operators of physical field equations on geometric meshes from a function-space perspective. We reveal that Hodge orthogonality fundamentally resolves spectral interference by isolating unlearnable topological degrees of freedom from learnable geometric dynamics, enabling an additive approximation confined to structure-preserving subspaces. Building on Hodge theory and operator splitting, we derive a principled operator-level decomposition. The result is a Hybrid Eulerian-Lagrangian architecture with an algebraic-level inductive bias we call Hodge Spectral Duality (HSD). In our framework, we use discrete differential forms to capture topology-dominated components and an orthogonal auxiliary ambient space to represent complex local dynamics. Our method achieves superior accuracy and efficiency on geometric graphs with enhanced fidelity to physical invariants.

Qingyu Yin, Yulun Wu, Zhennan Shen, Sunbowen Lee, Zhilin Wang, Yanshu Li, Chak Tou Leong, Jiale Kang, Jinjin Gu

We systematically evaluate Parameter-Efficient Fine-Tuning (PEFT) methods under the paradigm of Reinforcement Learning with Verifiable Rewards (RLVR). RLVR incentivizes language models to enhance their reasoning capabilities through verifiable feedback; however, while methods like LoRA are commonly used, the optimal PEFT architecture for RLVR remains unidentified. In this work, we conduct the first comprehensive evaluation of over 12 PEFT methodologies across the DeepSeek-R1-Distill families on mathematical reasoning benchmarks. Our empirical results challenge the default adoption of standard LoRA with three main findings. First, we demonstrate that structural variants, such as DoRA, AdaLoRA, and MiSS, consistently outperform LoRA. Second, we uncover a spectral collapse phenomenon in SVD-informed initialization strategies (e.g., PiSSA, MiLoRA), attributing their failure to a fundamental misalignment between principal-component updates and RL optimization. Furthermore, our ablations reveal that extreme parameter reduction (e.g., VeRA, Rank-1) severely bottlenecks reasoning capacity. We further conduct ablation studies and scaling experiments to validate our findings. This work provides a definitive guide for advocating for more exploration for parameter-efficient RL methods.

General Machine Learning · Transfer, Multitask and Meta-learning

Yayuan Li, Ze Peng, Jian Zhang, Jintao Guo, Yue Duan, Yinghuan Shi

Model merging combines multiple fine-tuned models into a single model by $\textit{adding}$ their weight updates, providing a lightweight alternative to retraining. Existing methods primarily target resolving conflicts between task updates, leaving the failure mode of over-counting shared knowledge unaddressed. We show that when tasks share aligned spectral directions (\ie, overlapping singular vectors), a simple linear combination repeatedly accumulates these directions, inflating the singular values and biasing the merged model toward shared subspaces. To mitigate this issue, we propose Singular Value Calibration (SVC), a training-free and data-free post-processing method that quantifies subspace overlap and rescales inflated singular values to restore a balanced spectrum. Across vision and language benchmarks, SVC consistently improves strong merging baselines and achieves state-of-the-art performance. Furthermore, by modifying only the singular values, SVC improves the performance of Task Arithmetic by 13.0\%.

Deep Learning · Graph Neural Networks

Maysam Behmanesh, Erkan Turan, Maks Ovsjanikov

Graph alignment, the problem of identifying corresponding nodes across multiple graphs, is fundamental to numerous applications. Most existing unsupervised methods embed node features into latent representations to enable cross-graph comparison without ground-truth correspondences. However, these methods suffer from two critical limitations: the degradation of node distinctiveness due to oversmoothing in GNN-based embeddings, and the misalignment of latent spaces across graphs caused by structural noise, feature heterogeneity, and training instability, ultimately leading to unreliable node correspondences. Our key insight is that rather than balancing these objectives purely in the feature space, a unified framework can simultaneously filter in both feature and map spaces. We propose a novel framework that employs a dual-pass encoder to inject high-frequency discriminability into node features, paired with a geometry-aware functional map module that operates on the correspondence itself. This functional map module learns bijective and isometric transformations that align latent spaces while acting as a low-pass filter on correspondences, enforcing smoothness and robustness as a structural prior in the map space. Extensive experiments on graph benchmarks demonstrate that our method consistently outperforms existing unsupervised alignment baselines, exhibiting superior robustness to structural inconsistencies and challenging alignment scenarios.

General Machine Learning · Evaluation

Yueqi Zhang, Jin Hu, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yiwei Li, Jiayi Shi, Chuyi Tan, Ji Zhang, Boyuan Pan 等

The prohibitive cost of evaluating Large Language Models (LLMs) necessitates efficient alternatives to full-scale benchmarking. Prevalent approaches address this by identifying a small coreset of items to approximate full-benchmark performance. However, existing methods must estimate a reliable item profile from response patterns across many source models, which becomes statistically unstable when the source pool is small. This dependency is particularly limiting for newly released benchmarks with minimal historical evaluation data. We argue that discrete correctness labels are a lossy view of the model's decision process and fail to capture information encoded in hidden states. To address this, we introduce \textsc{RepCore}, which aligns heterogeneous hidden states into a unified latent space to construct representative coresets. Using these subsets for performance extrapolation, \textsc{RepCore} achieves precise estimation accuracy with as few as ten source models. Experiments on five benchmarks and over 200 models show consistent gains over output-based baselines in ranking correlation and estimation accuracy. Spectral analysis further indicates that the aligned representations contain separable components reflecting broad response tendencies and task-specific reasoning patterns.