论文检索

Applications · Neuroscience, Cognitive Science

Subati Abulikemu, Tiago Azevedo, Michail Mamalakis, John Suckling

In network neuroscience, functional brain systems are often characterized using separate yet related graph-theoretic or spectral descriptors, overlooking how these properties covary and partially overlap across individuals and conditions. We anticipate that dense, weighted functional connectivity graphs lie on a low-dimensional latent geometry along which both topological and spectral structures vary smoothly at the population level. Although graph-based deep learning offers a powerful framework for modeling these brain connectomes, supervised approaches are constrained by the limited availability of labeled data. Existing unsupervised graph representation methods also typically focus on node-level embeddings, which are limited in capturing compact graph-level representations that preserve information from dense functional connectomes. To address these gaps, we learn compact brain graph representations using a graph transformer autoencoder, where domain-specific, aligned functional gradient geometry provides an inductive bias to guide learning. Despite being trained in a fully unsupervised manner, our approach meaningfully separates cognitive states and enables decoding of visual stimuli, with performance further improved by incorporating neural dynamics. In parallel, to enable generation of synthetic brain graphs, we fit a diffusion model to the learned latent representation and decode samples back to dense connectomes.

Deep Learning · Algorithms

Shenghao Yang, Zhichao Wang, Oleg Balabanov, N. Benjamin Erichson, Michael Mahoney

Matrix functions such as square root, inverse roots, and orthogonalization play a central role in preconditioned gradient methods for neural network training. This has motivated the development of iterative algorithms that avoid explicit eigendecompositions and rely primarily on matrix multiplications, making them well suited for modern GPU accelerators. We present PRISM (Polynomial-fitting and Randomized Iterative Sketching for Matrix functions computation), a general framework for accelerating iterative algorithms for computing matrix functions. PRISM combines adaptive polynomial approximation with randomized sketching: at each iteration, it fits a polynomial surrogate to the current spectrum via a sketched least-squares problem, adapting to the instance at hand with minimal overhead. We apply PRISM to accelerate Newton–Schulz-like iterations for matrix square roots and orthogonalization, which are core primitives in machine learning. Unlike prior methods, PRISM requires no explicit spectral bounds or singular value estimates; it adapts automatically to the evolving spectrum. Empirically, PRISM accelerates training when integrated into Shampoo and Muon optimizers.

Theory · Learning Theory

Jiaolong Wang, Fode Zhang, Lingrui Wang

Reliable estimation of Amari $\alpha$-divergences underpins variational inference, yet unconstrained neural critics are notoriously prone to instability. We propose a scalable estimator by constraining the critic to a Reproducing Kernel Hilbert Space (RKHS) ball and approximating the kernel via band-limited Random Fourier Features (RFF). This formulation yields a linear-time objective amenable to mini-batch stochastic optimization while avoiding the cubic complexity of Gram-matrix methods. We present a unified analysis based on a four-term error decomposition—comprising RKHS approximation, feature discretization, statistical deviation, and optimization residual. Under a spectral source condition, we derive non-asymptotic bounds establishing that the RKHS approximation bias scales as $\mathcal{O}(R^{-\gamma})$, the RFF discretization error as $\mathcal{O}(R D^{-1/2})$, and the statistical error as $\mathcal{O}(R n^{-1/2})$. We further show that statistical non-degeneracy induces intrinsic local curvature, enabling our proposed Armijo-SGD to achieve local linear convergence. Empirical evaluations demonstrate that the RFF-RKHS estimator outperforms varying-representation baselines in stability, and applying this spectral regularization to GAN critics significantly enhances the capture of high-frequency data components.

Applications · Time Series

Xiangfei Qiu, Kangjia Yan, Xvyuan Liu, Xingjian Wu, Jilin Hu

Irregular multivariate time series (IMTS) forecasting is challenging due to non-uniform sampling and variable asynchronicity. These irregularities violate the equidistant assumptions of standard models, hindering local temporal modeling and rendering classical frequency-domain methods ineffective for capturing global periodic structures. To address this challenge, we propose TFMixer, a joint time–frequency modeling framework for IMTS forecasting. Specifically, TFMixer incorporates a Global Frequency Module that employs a learnable Non-Uniform Discrete Fourier Transform (NUDFT) to directly extract spectral representations from irregular timestamps. In parallel, the Local Time Module introduces a query-based patch attention mechanism to adaptively aggregate informative temporal segments and alleviate information density imbalance. Finally, TFMixer fuses the time-domain and frequency-domain representations to generate forecasts and further leverages inverse NUDFT for explicit seasonal extrapolation. Extensive experiments on real-world IMTS benchmarks demonstrate the effectiveness and robustness of TFMixer under irregular sampling and missing data.

Applications · Computer Vision

Xuanhao Qi, Tom Luan, Yukang Zhang, Jinkai Zheng, su zhou, Shuwei Li, Lei Tan

Despite significant progress in multi-modal Re-Identification (ReID), existing methods tend to emphasize low-frequency cues. Consequently, they focus on attributes such as color, illumination, and coarse appearance, while overlooking mid- and high-frequency structures that encode geometric, textural, and identity-discriminative details. This imbalance leads to incomplete spectral representations and unstable cross-modal alignment. To overcome these limitations, we introduce FUSE, a frequency-domain framework that reformulates multi-modal ReID as a two-stage process of spectral disentanglement and energy alignment. The proposed Spectral Decomposition Module (SDM) adaptively partitions features into low, mid, and high-frequency subspaces, enabling hierarchical spectral modeling. The Cross-Modal Alignment Module (CAM) further enforces energy alignment and subspace complementarity across modalities via frequency-consistency regularization. In addition, FUSE incorporates learnable frequency modulation to enhance robustness under varying illumination and heterogeneous sensor conditions. Extensive experiments on RGBNT201, RGBNT100, and MSVR310 show that FUSE achieves 9.1% mAP and 9.5% Rank-1 improvements, establishing an interpretable frequency-domain paradigm for multi-modal representation learning.

Deep Learning · Graph Neural Networks

Ruiting Zhao, Ming Li, Lixin Cui, Lu Bai, Feilong Cao, Ke Lv, Pietro Lió

Hypergraph contrastive learning is an effective paradigm for representation learning on higher-order relational data, yet existing methods largely ignore that hyperedges link nodes with multi-level semantics. Standard contrastive objectives emphasize instance discrimination via hyperspherical uniformity and tend to push embeddings apart in an indiscriminate manner. We show that this leads to a *Hierarchy–Uniformity Conflict*, whose geometric manifestation is *Semantic Flattening*, where the semantic depth of hyperedges collapses into a nearly flat cloud of instances. To address this issue, we introduce **HyperDepth**, a hypergraph contrastive learning framework that moves representations towards a hierarchy–uniformity equilibrium by jointly coordinating spectral and geometric signals. HyperDepth employs a decoupled spectral encoding scheme with adaptive gating so that high-frequency components focus on local instance discrimination while low-frequency components capture global hierarchical structure. On top of this, an energy-based hierarchical Alignment module attaches a learnable prototype tree to the representation space and minimizes an interpretable energy functional to recover the semantic depth of hyperedges. Theoretically, under a mild frequency-separation assumption, we show that the local contrastive and global hierarchical objectives operate on orthogonal spectral components and admit equilibrium embeddings that preserve semantic depth while still retaining instance-level discrimination. Experiments on 15 hypergraph datasets and 17 supervised and self-supervised baselines, spanning homophilic and heterophilic regimes, show that HyperDepth attains strong performance with the best average rank.

Ruiting Zhao, Ming Li, Lixin Cui, Lu Bai, Feilong Cao, Ke Lv, Pietro Lió

Hypergraph contrastive learning is an effective paradigm for representation learning on higher-order relational data, yet existing methods largely ignore that hyperedges link nodes with multi-level semantics. Standard contrastive objectives emphasize instance discrimination via hyperspherical uniformity and tend to push embeddings apart in an indiscriminate manner. We show that this leads to a *Hierarchy–Uniformity Conflict*, whose geometric manifestation is *Semantic Flattening*, where the semantic depth of hyperedges collapses into a nearly flat cloud of instances. To address this issue, we introduce **HyperDepth**, a hypergraph contrastive learning framework that moves representations towards a hierarchy–uniformity equilibrium by jointly coordinating spectral and geometric signals. HyperDepth employs a decoupled spectral encoding scheme with adaptive gating so that high-frequency components focus on local instance discrimination while low-frequency components capture global hierarchical structure. On top of this, an energy-based hierarchical Alignment module attaches a learnable prototype tree to the representation space and minimizes an interpretable energy functional to recover the semantic depth of hyperedges. Theoretically, under a mild frequency-separation assumption, we show that the local contrastive and global hierarchical objectives operate on orthogonal spectral components and admit equilibrium embeddings that preserve semantic depth while still retaining instance-level discrimination. Experiments on 15 hypergraph datasets and 17 supervised and self-supervised baselines, spanning homophilic and heterophilic regimes, show that HyperDepth attains strong performance with the best average rank.

Deep Learning · Graph Neural Networks

Xiang Wu, Rong-Hua Li, Xunkai Li, Kangfei Zhao, Hongchao Qin, Guoren Wang

Graph coarsening reduces the size of a graph while preserving certain properties. Most existing methods preserve either spectral or spatial characteristics. Recent research has shown that preserving topological features helps maintain the predictive performance of graph neural networks (GNNs) trained on the coarsened graph but suffers from exponential time complexity. To address these problems, we propose a Scalable Topology-Preserving Graph Coarsening (STPGC) by introducing the concepts of graph strong collapse and graph edge collapse extended from algebraic topology. STPGC comprises three new algorithms, *GStrongCollapse*, *GEdgeCollapse*, and *NeighborhoodConing* based on these two concepts, which eliminate dominated nodes and edges while rigorously preserving topological features. We further prove that STPGC preserves the GNN receptive field and develop approximate algorithms to accelerate GNN training. Experiments in node classification of GNNs demonstrate the efficiency and effectiveness of STPGC.

General Machine Learning · Unsupervised and Semi-supervised Learning

Joonhyuk Lee, Virginia L., Sarah Zhao, Yash Nair, Asher Spector, Regev Cohen, Emmanuel J Candes

Verification of model outputs is rapidly emerging as a key primitive for both training and real-world deployment of large language models (LLMs). In practice, this often involves using imperfect LLM judges and reward models since ground truth acquisition can be time-consuming and expensive. We introduce Fully Unsupervised Score Ensembling (FUSE), a method for improving verification quality by ensembling verifiers without access to ground truth correctness labels. The key idea behind FUSE is to control conditional dependencies between verifiers in a manner that improves the unsupervised performance of a class of spectral algorithms from the ensembling literature. Despite requiring zero ground truth labels, FUSE typically matches or improves upon semi-supervised alternatives in test-time scaling experiments with diverse sets of generator models, verifiers, and benchmarks. In particular, we validate our method on both conventional academic benchmarks such as GPQA Diamond and on frontier, unsaturated benchmarks such as Humanity's Last Exam and IMO Shortlist questions.

Deep Learning · Generative Models and Autoencoders

Zhao zuopeng, Ying Liu, xiaoyu Li, SuLuo, Lu Li, Wen L

Existing diffusion models have made significant progress in generating realistic images. However, their direct adaptation to remote sensing imagery often disregards intrinsic physical laws. This oversight frequently leads to spectral distortion and radiometric inconsistency, severely limiting the scientific utility of generated data. To address this issue, this paper introduces AnyBand-Diff, a novel spectral-prior-guided diffusion framework tailored for robust spectral reconstruction. Specifically, we design a Masked Conditional Diffusion backbone integrated with a dual stochastic masking strategy, empowering the model to recover complete spectral information from arbitrary band subsets. Subsequently, to ensure radiometric fidelity, a Physics-Guided Sampling mechanism is proposed, leveraging gradients from a differentiable physical model to explicitly steer the denoising trajectory toward the manifold of physically plausible solutions. Furthermore, a Multi-Scale Physical Loss is formulated to enforce rigorous constraints across pixel, region, and global levels in a joint manner. Extensive experiments confirm the effectiveness of AnyBand-Diff in generating reliable imagery and achieving accurate spectral reconstruction, contributing to the advancement of physics-aware generative methods for Earth observation.

Deep Learning · Large Language Models

Valentin NOËL

Validating mathematical reasoning in large language models currently requires a trade-off between computationally expensive learned verifiers and the unreliability of output-based heuristics. We therefore propose a training-free, mechanistic alternative: spectral analysis of attention topology. By treating attention matrices as dynamic graphs over tokens, we extract four interpretable spectral diagnostics, Fiedler value, High-Frequency Energy Ratio (HFER), spectral entropy, and graph smoothness, that differentiate valid reasoning from hallucinated outputs without any learned parameters. We perform experiments across seven models from four architectural families (Llama, Qwen, Phi, Mistral) yield effect sizes up to Cohen's $d = 3.30$ ($p < 10^{-116}$), enabling $85$--$96\%$ classification accuracy with a single threshold. We discover that spectral analysis detects logical coherence rather than compiler acceptance: proofs rejected by formal verifiers due to timeouts or missing imports are correctly identified as valid, a phenomenon we term "Platonic validity". Furthermore, causal ablation studies confirm that this spectral signature reflects the functional health of induction head circuits, establishing a mechanistic basis for the method. We also identify an architectural dependency: Sliding Window Attention shifts the discriminative signal from HFER to late-layer smoothness ($d = 2.09$, $p < 10^{-48}$), demonstrating that attention mechanism design determines which spectral features capture reasoning validity. The method generalizes to informal chain-of-thought reasoning ($d = 0.78$, $p < 10^{-3}$). These findings establish spectral graph analysis as a principled framework for reasoning verification, with immediate applications to hallucination detection and real-time safety monitoring.

Deep Learning · Self-Supervised Learning

Denis Janiak, Jakub Binkowski, Tomasz Kajdanowicz

Out-of-distribution (OOD) detection is critical for reliable deployment of vision models. Mahalanobis-based detectors remain strong baselines, yet their performance varies widely across modern pretrained representations, and it is unclear which properties of a feature space cause these methods to succeed or fail. We conduct a large-scale study across diverse foundation-model backbones and Mahalanobis variants. First, we show that Mahalanobis-style OOD detection is not universally reliable: performance is highly representation-dependent and can shift substantially with pretraining data and fine-tuning regimes. Second, we link this variability to in-distribution geometry and identify a two-term ID summary that consistently tracks Mahalanobis OOD behavior across detectors: within-class spectral structure and local intrinsic dimensionality. Finally, we treat normalization as a geometric control mechanism and introduce radially scaled $\ell_2$ normalization, $\phi_\beta(z)=z/\|z\|^\beta$, which preserves directions while contracting or expanding feature radii. Varying $\beta$ changes the radii while preserving directions, so the same quadratic detector sees a different ID geometry. We choose $\beta$ from ID-only geometry signals and typically outperform fixed normalization baselines.

Deep Learning · Large Language Models

Xiaowen Jiang, Andrei Semenov, Sebastian Stich

We identify two empirical issues in large language model (LLM) training: (i) optimizer updates can have large spectral norms, potentially destabilizing training and degrading generalization; (ii) stochastic gradient noise can exhibit sparse spectral spikes, with a few dominant singular values much larger than the rest. We propose *SPECTRA*, a general framework addressing these by (i) *post*-spectral clipping of updates to enforce spectral-norm constraints (ii) optional *pre*-spectral clipping of gradients to suppress spectral noise spikes. We prove that post-clipping constitutes a Composite Frank-Wolfe method with spectral-norm constraints and weight regularization, recovering Frobenius and $\ell_{\infty}$-norm regularization with SGD-based and sign-based methods. We further analyze how pre-clipping mitigates sparse spectral spikes. We propose efficient soft spectral clipping via Newton-Schulz iterations, avoiding expensive SVD. Experiments on LLM pretraining show SPECTRA uniformly improves validation loss for various optimizers, including AdamW, Signum, and AdEMAMix, with the best-performing variants achieving state-of-the-art results. Models trained with SPECTRA exhibit smaller weight norms, confirming the link between spectral clipping and regularization.

Binwu Wang, Gaoyun Lin, Jiaming Ma, Qihe Huang, Zhengyang Zhou, Xu Wang, Pengkun Wang, Yang Wang

Multivariate time series (MTS) forecasting critically depends on modeling inter-variable dependencies, yet existing paradigms face a trade-off: channel-isolation strategies can suffer from information fragmentation in strongly coupled systems, whereas channel-interaction methods often introduce spurious interactions among irrelevant variables. To address this challenge, we propose Coherent Resonance Interaction with Spectral Priors (Crisp). Crisp adopts the principle that effective information exchange should occur only between variables with compatible oscillatory patterns. Concretely, we derive spectral priors in the frequency domain to construct dynamic resonance topologies. With a differentiable, adaptive, and strictly sparse blocking mechanism, Crisp forces attention weights for spectrally inconsistent neighbors to be exactly zero. In addition, we introduce a spectral-gated feature filtering module to refine variable representations using intrinsic spectral characteristics. Extensive experiments demonstrate that Crisp significantly outperforms 20+ baselines. Our code is available at Anonymous GitHub.

Applications · Chemistry, Physics, and Earth Sciences

Xiaohan Qin, Wenjie Du, Yang Wang

Identifying molecular structures from spectral data is essential for early-stage chemical analysis, yet it remains a difficult task due to the imbalance in functional group distributions. Current methods often overfit to prevalent groups while neglecting underrepresented ones, failing to capture key dependencies between functional groups. This highlights the need for a unified approach that addresses both data imbalance and structural constraints. In this work, we present SymSpectra, a Symmetric Conditional Information Bottleneck (SCIB) framework designed to seamlessly integrate multi-modal Spectra features. Our model employs the SCIB framework to fuse multi-modal spectroscopic data into a unified representation, effectively preserving discriminative signals while mitigating redundancy. To enhance robustness against data imbalance, we incorporate conditional mutual information into the training objective, increasing the model’s sensitivity to rare functional groups and challenging molecular cases. Additionally, a specialized module captures the dependencies among functional groups, improving both prediction accuracy and chemically meaningful interpretability. Experiments on multimodal spectral datasets demonstrate that SymSpectra significantly outperforms state-of-the-art methods, achieving an F1-score of 0.970 in substructure classification. More importantly, SymSpectra consistently outperforms baselines under various imbalanced scenarios, exhibiting superior robustness and generalizability, which may help advance the automation of chemical discovery. Our code can be found at https://anonymous.4open.science/r/SymSpectra-0017.

Pingzhu Liu, Chunming He, Zunnan Xu, Chao Hao, Bo Zhao, Xingyu Shao, Jun Zhou, Zitong YU, Xiu Li

Unsupervised Camouflaged Object Detection (UCOD) aims to identify objects concealed in their surroundings without relying on pixel-level labels. Existing methods rely solely on simple post-processing of DINO high-dimensional features to generate pseudo labels for training. However, these methods suffer from two major limitations: 1) pseudo labels they easily generate contain excessive noise, causing the model to learn substantial incorrect information. 2) Although pseudo-label supervision allows the model to understand the task, it remains insufficient for generating fine-grained segmentation of the camouflaged objects. To address these issues, we propose DualUCOD, a novel UCOD method based on dual-branch contrastive learning that effectively detects camouflaged objects without pixel-level labels. Specifically, we propose the Dual-Eigenvector Spectral Pseudo-Labeling (DESPL) strategy, which fuses semantic and color cues into an affinity matrix. We then compute the eigenvectors of its normalized graph Laplacian and generate high-quality pseudo-labels using these eigenvectors. Furthermore, we introduce a Boundary-Guided Foreground-Background Refinement (BGFBR) module that explicitly incorporates boundary information to improve segmentation accuracy. Finally, we introduce a Dual-Branch Contrastive Learning (DBCL) module that constructs positive and negative pairs from the original and augmented images, aligning positive representations while contrasting them against negatives to enhance camouflaged object understanding. Extensive experiments demonstrate that DualUCOD outperforms state-of-the-art methods on different datasets in the unsupervised setting.

Applications · Language, Speech and Dialog

Harshvardhan Takawale, Nirupam Roy, C. Phillip Brown

Neural acoustic fields often model time-domain impulse responses, which struggle to capture the frequency-selective wave behaviors that dominate confined, resonant environments. To address this, we propose INFER (Implicit Neural Frequency Response fields), a framework that directly learns continuous, complex-valued frequency response fields. Unlike prior time-domain methods, our frequency-first approach enables three key innovations: (1) end-to-end learning of frequency-specific attenuation and phase delay in 3D space; (2) a physics-based Kramers–Kronig consistency constraint that causally regularizes attenuation and phase delay; and (3) perceptual and hardware-aware spectral supervision that prioritizes critical auditory bands. We evaluate INFER across diverse settings, ranging from standard room-scale benchmarks (MeshRIR, RAF) to challenging, highly reverberant environments like real car cabins. Our approach significantly outperforms time- and hybrid-domain baselines, reducing average magnitude and phase reconstruction errors by over 39\% and 51\%, respectively, demonstrating state-of-the-art accuracy in modeling complex acoustic spaces.

General Machine Learning · Causality

Matteo Tusoni, Giuseppe Masi, Andrea Coletta, Aldo Glielmo, Viviana Arrigoni, Novella Bartolini

Exploring causal relationships in stochastic time series is a challenging yet crucial task with a vast range of applications, including finance, economics, neuroscience, and climate science. Many algorithms for Causal Discovery (CD) have been proposed; however, they often exhibit a high sensitivity to noise, resulting in spurious causal inferences on real data. In this paper, we observe that the frequency spectra of many real-world time series follow a power-law distribution, notably due to an inherent self-organizing behavior. Leveraging this insight, we build a robust CD method based on the extraction of power‑law spectral features that amplify genuine causal signals. Our method consistently outperforms state-of-the-art alternatives on both synthetic benchmarks and real-world datasets with known causal structures, demonstrating its robustness and practical relevance.

Probabilistic Methods · Everything Else

Tanish Agarwal, Kaustubh Shivshankar Shejole, Arpit Agarwal

Item response theory (IRT) is widely used in areas such as recommender systems, education, psychology, and other fields. A popular model for IRT is the Rasch model. Under this model, if a user with ability $\theta$ performs a task with difficulty $\beta$ then its label $X \sim \text{Bernoulli} (1 / (1 + \exp(-(\theta - \beta)))$. Existing joint maximum likelihood estimation approaches for this problem do not perform well on small datasets and also lack theoretical guarantees. Recently, Nguyen and Zhang proposed a two step approach: (1) spectral method for estimation of task parameters, (2) likelihood optimization for estimation of user parameters. While this approach is theoretically sound, it is not computationally efficient. In this work, we propose an EM-based algorithm for joint estimation of item and user parameters by introducing Pólya-Gamma latent variables, which simplify the logistic log-likelihood. We show that our algorithm is both theoretically sound and consistently outperforms existing methods on synthetic and real-world datasets.

Deep Learning · Generative Models and Autoencoders

Amir Dellali, Luca Lanzendörfer, Florian Grötschla, Roger Wattenhofer

We propose SALSA-V, a multimodal video-to-audio generation model capable of synthesizing highly synchronized, high-fidelity long-form audio from silent video content. Our approach introduces a masked diffusion objective, enabling audio-conditioned generation and the seamless synthesis of audio sequences of unconstrained length. Additionally, by integrating a shortcut loss into our training process, we achieve rapid generation of high-quality audio samples in as few as eight sampling steps, paving the way for near-real-time applications without requiring dedicated fine-tuning or retraining. We demonstrate that SALSA-V significantly outperforms existing state-of-the-art methods in both audiovisual alignment and synchronization with video content in quantiative evaluation and a human listening study. Furthermore, our use of random masking during training enables our model to match spectral characteristics of reference audio samples, broadening its applicability to professional audio synthesis tasks such as Foley generation and sound design.