论文检索

Deep Learning · Large Language Models

Paul Janson, Edouard Oyallon, Eugene Belilovsky

Foundation models have achieved remarkable success, yet their growing parameter counts pose significant computational and memory challenges. Low-rank factorization offers a promising route to reduce training and inference costs, but the community lacks a stable recipe for training models from scratch using exclusively low-rank weights while matching performance of the dense model. We demonstrate that Large Language Models (LLMs) can be trained from scratch using exclusively low-rank factorized weights for all non-embedding matrices without auxiliary "full-rank" guidance required by prior methods. While native low-rank training often suffers from instability and loss spikes, we identify uncontrolled growth in the spectral norm (largest singular value) of the weight matrix update as the dominant factor. To address this, we introduce **Spectron: Spectr**al renormalization with orthogonalizati**on**, which dynamically bounds the resultant weight updates based on the current spectral norms of the factors. Our method enables stable, end-to-end factorized training with negligible overhead. Finally, we establish compute-optimal scaling laws for natively low-rank transformers, demonstrating predictable power-law behavior and improved inference efficiency relative to dense models. Our code is available at [https://anonymous.4open.science/r/spectron-FB27](https://anonymous.4open.science/r/spectron-FB27)

Deep Learning · Graph Neural Networks

Samuel Fernandez, Eduardo Pavez, Antonio Ortega

Despite their theoretical advantages, spectral methods based on the graph Fourier transform (GFT) are seldom used in graph neural networks (GNNs) due to the cost of computing the eigenbasis and the lack of vertex-domain locality in spectral representations. As a result, most GNNs rely on local approximations such as polynomial Laplacian filters or message passing, which limit their ability to model long-range dependencies. In this paper, we introduce a novel factorization of the GFT into operators acting on subgraphs, which are then combined via a sequence of Cauchy matrices. We use this factorization to propose a new class of spectral GNNs, which we term L2G-Net (Local-to-Global Net). Unlike existing spectral methods, which are either fully global (when they use the GFT) or local (when they use polynomial filters), L2G-Net operates by processing the spectral representations of subgraphs and then combining them via structured matrices. Our algorithm avoids full eigendecompositions, exploiting graph topology to construct the factorization with quadratic complexity in the number of nodes, scaled by the subgraph interface size. Experiments on benchmarks stressing non-local dependencies show that L2G-Net outperforms existing spectral techniques and is competitive with the state-of-the-art with orders of magnitude fewer learnable parameters.

Bowen Zhao, Qianqian Wang

Deep hypergraph clustering has shown strong potential in learning node representations by modeling high-order relationships. However, most existing methods rely on fixed propagation mechanisms that implicitly assume homophily, where connected nodes tend to be similar. This assumption often fails in real-world scenarios, especially in heterophilic settings, leading to degraded clustering performance. To bridge this gap, we propose a \textbf{Uni}fied Framework for \textbf{D}eep \textbf{H}ypergraph \textbf{C}lustering (Uni-DHC). Specifically, we introduce a learnable high-order hypergraph propagation scheme that aggregates information from multiple propagation orders and adaptively learns their importance from data. To stabilize unsupervised training and prevent structural redundancy introduced by high-order aggregation, we further impose consistency at the node level and decorrelation at the hyperedge level. From a spectral perspective, we show that conventional HGNN-style propagation corresponds to a fixed low-pass filter, while the proposed approach induces a learnable polynomial spectral filter. Extensive experiments on homophilic and heterophilic benchmarks demonstrate that Uni-DHC consistently outperforms state-of-the-art methods, with particularly strong gains in heterophilic settings.

Jiachen Liu

Spectral clustering, a widely-used technique for graph-based data partitioning, faces a severe computational bottleneck due to its $O(n^{3})$ time complexity. While anchor-based approximations reduce the complexity to $O(nm^{2})$ ($m \ll n$), they often yield degenerate solutions in the absence of explicit cluster-size control. To address this limitation, we propose \textbf{SC-FAGC (Size-Constrained Fast Anchor Graph Clustering)}, a unified formulation that integrates entropy regularization and bilateral cardinality constraints within an anchor-based spectral clustering framework. Our model simultaneously promotes cohesive clusters and enforces soft lower and upper bounds on cluster sizes, thus avoiding trivial or highly unbalanced partitions. To solve the resulting non-convex optimization problem efficiently, we develop an \textbf{Iteratively Re-weighted (IRW)} optimization scheme, which sequentially linearizes the objective and solves each subproblem via a \textbf{Double-Bounded Optimal Transport (DB-OT)} solver based on the \textbf{Sinkhorn--Knopp} algorithm. This approach guarantees convergence while maintaining scalability. Extensive experiments on benchmark datasets demonstrate that SC-FAGC consistently achieves state-of-the-art performance in terms of accuracy, purity, and recall, while strictly satisfying the prescribed cluster-size constraints. Thus the proposed method offers a principled and scalable solution for large-scale graph clustering with controllable partition structure.

Applications · Computer Vision

Xijie Xiang, Lin Zhu, Yonghong Tian

Autofocus for spike cameras is challenging because their sparse binary measurements do not provide reliable instantaneous gradients, and noise or illumination drift often breaks the unimodal assumptions behind conventional focus measures. We show that during a focus sweep, the stable sensor-observable cue is a persistent migration of spectral energy in the frequency domain: energy shifts outward toward higher frequencies when approaching focus and recedes under renewed defocus. Building on this observation, we propose CEN (Centroid-based Energy Navigation), a frequency-domain autofocus method that measures spectral migration via a bounded spectral centroid computed on accumulated spike blocks, without image reconstruction or explicit edge extraction. To handle multi-peak and irregular responses in real scenes, CEN further performs structure-consistent response identification, selecting the frequency bound whose curve exhibits a clear, localized, interior extremum, followed by robust peak localization using a weighted near-maximum centroid. Experiments on spike-camera dataset demonstrate that CEN achieves the best overall accuracy and response discriminability across diverse scenes, motion types, and illumination variation patterns.

Applications · Computer Vision

Xingyuan Li, HaoYuan Xu, Shulin Li, Xiang Chen, Zhiying Jiang, Jinyuan Liu

Infrared and visible video fusion is essential for achieving comprehensive perception in dynamic scenes. However, maintaining temporal consistency remains a formidable challenge. Conventional methods relying on optical flow often suffer from geometric rigidity and ghosting artifacts. Moreover, standard diffusion-based fusion models typically operate in a frame-by-frame manner; when extended to autoregressive settings, they lack intrinsic temporal constraints and are prone to severe error accumulation and drifting, where minor artifacts amplify over time. To address these limitations, we propose a drift-resilient video fusion method that reformulates the task as history-conditioned motion generation. We introduce Stabilized History Guidance and Soft Temporal Anchoring to reframe temporal consistency as spectral filtering, implicitly aggregating motion dynamics without rigid alignment. Furthermore, our Decoupled Structure-Motion Adaptation strategy bridges pre-trained priors and structural constraints via two-stage training and latent refinement. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both fusion quality and temporal stability.

Applications · Computer Vision

Wudi Chen, Zhiyuan Zha, Xin Yuan, Shigang Wang, Bihan Wen, Jiantao Zhou, Gang Yan, zipei fan, Ce Zhu

Recent advances have demonstrated that coded aperture snapshot spectral imaging (CASSI) systems show great potential for capturing 3D hyperspectral images (HSIs) from a single 2D measurement. Despite the inherent spectral continuity of scenes captured by CASSI, most existing reconstruction methods are restricted to fixed, discrete spectral outputs, thereby precluding continuous spectral reconstruction or spectral super-resolution. To address this challenge, we propose Phy-CoSF, which synergizes deep unfolding networks with implicit neural representations, establishing a new paradigm for continuous spectral reconstruction and super-resolution in CASSI. Specifically, we propose a two-phase architecture that bridges discrete-wavelength training with continuous spectral rendering, enabling the synthesis of high-fidelity HSIs at arbitrary target wavelengths. At the core of our framework lies the continuous spectral fields (CoSF) module, embedded within each unfolding stage as a dynamic prior, which comprises a triple-branch cross-domain feature mixer for comprehensive spatial–frequency–channel feature fusion, alongside a spectral synthesis head that generates spectral intensities by querying continuous wavelength coordinates. Extensive experimental results demonstrate that Phy-CoSF not only achieves continuous modeling at arbitrary spectral resolutions but also outperforms many state-of-the-art methods in both reconstruction fidelity and spectral detail preservation.

Deep Learning · Graph Neural Networks

Wentao Yu, Sheng Wan, Ge Gao, Bo Han, Chen Gong

Graph Federated Learning (GFL) enables distributed graph representation learning while protecting the privacy of graph data. However, GFL suffers from heterogeneity arising from diverse node features and structural topologies across multiple clients. To address both types of heterogeneity, we propose a novel graph Federated learning method via Semantic and Structural Alignment (FedSSA), which shares the knowledge of both node features and structural topologies. For node feature heterogeneity, we propose a novel variational model to infer class-wise node distributions, so that we can cluster clients based on inferred distributions and construct cluster-level representative distributions. We then minimize the divergence between local and cluster-level distributions to facilitate semantic knowledge sharing. For structural heterogeneity, we employ spectral Graph Neural Networks (GNNs) and propose a spectral energy measure to characterize structural information, so that we can cluster clients based on spectral energy and build cluster-level spectral GNNs. We then align the spectral characteristics of local spectral GNNs with those of cluster-level spectral GNNs to enable structural knowledge sharing. Experiments on six homophilic and five heterophilic graph datasets under both non-overlapping and overlapping partitioning settings demonstrate that FedSSA consistently outperforms eleven state-of-the-art methods. Our code is available at https://anonymous.4open.science/r/FedSSA.

Deep Learning · Graph Neural Networks

Chen Zhu, YAYING ZHANG

Graph anomaly detection (GAD) is a fundamental task in graph learning. However, most existing methods rely on the homophily assumption, which posits that connected nodes tend to share the same labels. This assumption often fails in the presence of edge heterophily, leading to degraded performance. We first observe that down-weighting heterophilic edges, relative to the original or randomly weighted graphs, results in a more concentrated spectral energy distribution, thereby facilitating the learning of discriminative spectral embeddings. Moreover, existing methods typically embed graphs in Euclidean spaces, neglecting the importance of heterophily in manifold spaces. Motivated by these observations, we propose HSMAD, a novel framework for GAD. It consists of two key components: the Heterophily-Weighted Spectral Filtering module, which reconstructs the Laplacian using heterophily-based edge weighting for spectral filtering, and the Heterophily-Routed Manifold Update module, which routes neighborhood messages to the appropriate manifold for node feature updates, enabling curvature-adaptive representation learning. These spectral and geometric representations are jointly leveraged for anomaly detection. Extensive experiments on six real-world datasets show that HSMAD achieves state-of-the-art performance across the average F1-Macro, AUROC, AUPRC, and G-Mean. Specifically, the average F1-Macro score improves by 2.66% over the best-performing method.

Applications · Health / Medicine

Zeyu An, Wanyu LIN

{\em De novo} peptide sequencing is a pivotal technique that directly reconstructs amino acid sequences from tandem mass spectrometry (MS/MS) data; it enables the identification of novel proteins and variants absent from reference databases. Previous methods are typically based on autoregressive (AR) decoding or one-shot generation. The AR-based methods conflict with the bidirectional and globally constrained nature of MS/MS evidence and inevitably accumulate errors, while one-shot generation does not explicitly enforce physical constraints, failing to produce chemically valid and reliable peptides in a single pass. Accurate sequencing necessitates reasoning over the entire peptide simultaneously, enabling iterative self-correction under global constraints. To this end, we introduce $\textbf{PhysNovo}$, a novel paradigm that harnesses discrete diffusion to enable simultaneous global reasoning and iterative refinement. Specifically, PhysNovo reformulates sequencing as a $\textbf{phys}$ically mass-constrained reasoning process by embedding a knapsack-based feasibility kernel to enforce exact precursor mass consistency. By conditioning the diffusion process on global spectral context, PhysNovo supports abductive reasoning where bidirectional evidence is exploited to iteratively resolve local inconsistencies and ensure physically valid predictions. PhysNovo achieves state-of-the-art performance, exceeding baselines by over 2\% in precision, with larger gains on out-of-distribution data.

Deep Learning · Generative Models and Autoencoders

Yuehao Wang, Peihao Wang, Hanwen Jiang, Ziyi Yang, Qixing Huang, Zhangyang “Atlas” Wang

Diffusion models have shown remarkable performance on diverse generation tasks. Recent work finds that imposing representation alignment on the hidden states of diffusion networks can both facilitate training convergence and enhance sampling quality, yet the mechanism driving this synergy remains insufficiently understood. In this paper, we investigate the connection between self-supervised spectral representation learning and diffusion generative models through a shared perspective on perturbation kernels. On the diffusion side, samples (e.g., images, videos) are produced by reversing a stochastic noise-injection process specified by Gaussian kernels; on the spectral representation side, spectral embeddings emerge from contrasting positive and negative relations induced by random perturbation kernels. Motivated by this, we propose a self-supervised spectral representation alignment method to facilitate diffusion model training. In addition, we clarify how joint spectral learning can benefit diffusion training from a geometric perspective. Furthermore, we find that the optimization of the spectral alignment objective is in an equivalent form of diffusion score distillation in the representation space. Building on these findings, we integrate a spectral regularizer into diffusion training objectives to improve the performance of diffusion models on multiple datasets. Experiments across images and 3D point clouds show consistent gains in generation quality.

Deep Learning · Theory

Torben Berndt, Jan Stuehmer

Equivariance is a powerful inductive bias in neural networks, improving generalisation and physical consistency. Recently, however, non-equivariant models have regained attention, due to their better runtime performance and imperfect symmetries that might arise in real-world applications. This has motivated the development of approximately equivariant models that strike a middle ground between respecting symmetries and fitting the data distribution. Existing approaches in this field either rely on sampling from a group, incurring a high sample complexity, or explicitly parameterise a model as a sum of an equivariant and non-equivariant network. This work instead approaches approximate equivariance via a projection-based regulariser which leverages a layer-wise orthogonal decomposition of a network's layers into equivariant and non-equivariant components. In contrast to existing methods, this penalises non-equivariance at an operator level across the full group orbit, rather than point-wise as in sample-based approaches. We present a mathematical framework for computing the non-equivariance penalty exactly and efficiently in both the spatial and spectral domains. In our experiments, our method consistently outperforms prior approximate equivariance approaches in both model performance and efficiency, achieving substantial runtime gains over sample-based regularisers.

Probabilistic Methods · Variational Inference

Yuhang Xi, Yu-Feng Yu, Chuan-Xian Ren, Zhao-Rong Lai

Parameter-Efficient Fine-Tuning (PEFT) is essential for adapting Large Language Models, yet existing methods typically struggle to balance model capacity with computational efficiency. Standard approaches often enforce rigid low-rank constraints, while dynamic alternatives incur significant memory overheads. To resolve this dilemma, we propose Spectral Bridge Variational Inference (SBVI), a geometric framework that reformulates LoRA not as static parameter optimization, but as a continuous Wasserstein gradient flow on the manifold of Gaussian measures. Rather than fixing the rank at initialization, SBVI governs the singular value evolution via a stochastic differential equation driven by a thermodynamic competition between task gradients and adaptive entropic friction. This mechanism induces a spectral bifurcation that automatically prunes redundant noise modes while amplifying signal-rich components, naturally discovering a layer-wise optimal rank distribution. We derive a scalable algorithm with linear complexity using factorized Riemannian retractions and an Empirical Bayes friction update. Experiments on reasoning and coding benchmarks demonstrate that SBVI achieves state-of-the-art performance, offering superior accuracy and memory efficiency compared to existing static and dynamic adaptation methods.

Optimization · Zero-order and Black-box Optimization

Sijia Liu, Yicheng Lang, Soumyadeep Pal, Changsheng Wang, Yancheng Huang, Chongyu Fan, James Diffenderfer, Bhavya Kailkhura, Yihua Zhang

Zeroth-order (ZO) optimization, learning from finite differences of function evaluations without backpropagation, has recently regained attention in deep learning due to its memory efficiency and applicability to gray- or black-box pipelines. Yet, ZO methods are often dismissed as fundamentally unscalable because of estimator variance and unfavorable query complexity. We argue that this conclusion might be misguided: ZO optimization is underexplored, not underpowered. We show that many perceived limitations stem from myopic development practices, most notably full-space, element-wise, estimator-centric designs. We articulate six positions spanning the algorithmic, systems, and evaluation stack. First, we revisit the feasibility boundaries of estimator-centric ZO methods through variance control, variance–query tradeoffs, and directional-derivative lenses. Then, we identify three underexplored opportunities: (i) subspace and spectral views of ZO that enable interpretable variance reduction with graceful query scaling, (ii) the forward-only nature of ZO as a systems advantage for communication-efficient, pipeline-friendly, and resource-constrained training, and (iii) the need to de-obfuscate ZO evaluations from task complexity. We strongly advocate rethinking ZO optimization around its unique strengths and acting accordingly, opening a viable path toward large-scale, system-aware, and resource-efficient learning with ZO optimization.

General Machine Learning · Causality

Saber Salehkaleybar

We study the problem of recovering the parameters of a multivariate Ornstein–Uhlenbeck (OU) process from steady-state observational and interventional data. In many applications, such as large-scale gene perturbation experiments, only stationary “snapshot” measurements are available, making standard stochastic differential equation estimation methods that rely on time-series trajectories inapplicable. We first establish an identifiability result: one intervention per strongly connected component (SCC) of the drift graph suffices to recover all OU process parameters generically up to a global scaling factor. This holds provided that the SCC condensation graph is connected with a single root and certain spectral nondegeneracy assumptions hold. We propose a recursive learning algorithm that orders SCCs topologically and, for each component, isolates its marginal dynamics and solves a linear system derived from the steady-state moment equations, leveraging parameters recovered for upstream components. Building on this theoretical foundation, we propose a regularized least-squares estimator that jointly minimizes residuals of the steady-state mean and covariance equations across observational and interventional data. Experiments on synthetic and real datasets demonstrate the effectiveness of our method in recovering parameters and predicting unseen interventions.

Haowen Li, Tianxiang Li, Yi Yang, Boyu Cao, Qi Liu

The advancement of diffusion-based text-to-music generation has opened new avenues for zero-shot music editing. However, existing methods fail to achieve context-aware editing, which requires altering specific stems while strictly preserving the background accompaniment. This limitation severely hinders practical application, since real-world production necessitates precise manipulation of components within dense mixtures. Our key finding is that, while vanilla cross-attention captures semantic intent, it lacks the spectral resolution to strictly localize targets in dense mixtures, leading to boundary leakage. To resolve this dilemma, we propose Polyphonia, a training-free editing framework with Acoustic-Informed Attention Calibration. Rather than relying solely on diffuse semantic attention, Polyphonia leverages a probabilistic acoustic prior to establish coarse boundaries, enabling background context preserved precise semantic synthesis. For evaluation, we propose PolyEvalPrompts, a standardized prompt set with 1,170 context-aware music editing tasks. Specifically, Polyphonia achieves an increase of 15.5% in target alignment compared to baselines, while maintaining competitive music fidelity and background integrity.

Zhaoyang Li, Yanjun Li, Wangkai Li, Yujia Chen, Tianzhu Zhang

Vision-Language Models (VLMs) are costly at inference time because they must process long sequences of visual tokens. Existing token pruning methods often degrade under high compression by blindly discarding information, breaking spatial structure or collapsing diversity. We propose SpecFlow, a training-free framework that shifts the paradigm from destructive pruning to conservative condensation, strictly enforcing spatial coverage and statistical conservation to ensure stability. Treating visual tokens as nodes in a $k$NN graph, SpecFlow (i) computes a stable importance field via spectral heat flow to preserve structural coherence, (ii) allocates budgets via adaptive spatial partitioning to guarantee coverage, and (iii) aggregates discarded information into coreset sinks to maintain statistical conservation. The method is plug-and-play, requires no fine-tuning, and is compatible with FlashAttention. Experiments confirm that our SpecFlow outperforms SOTA methods across tasks, VLM architectures, and pruning ratios. Notably, LLaVA-1.5 with SpecFlow retains 95.6\% of original performance despite pruning 88.9\% of visual tokens, offering an exceptional efficiency-accuracy balance.

General Machine Learning · Clustering

Gwendal Debaussart-Joniec, Harry Sevi, Matthieu Jonckheere, Argyris Kalogeratos

Vertex-level clustering for directed graphs (digraphs) remains challenging as edge directionality breaks the key assumptions underlying popular spectral methods, which also incur the overhead of eigen-decomposition. This paper proposes *Parametrized Power Iteration Clustering* (ParPIC), a random-walk-based clustering method for weakly connected digraphs. This builds over the Power-Iteration Clustering paradigm, which uses the rows of the iterated diffusion operator as a data embedding. ParPIC has three important features: the use of parametrized reversible random walk operators, the automatic tuning of the diffusion time, and the efficient truncation of the final embedding, which produces low-dimensional data representations and reduces complexity. Empirical results on synthetic and real-world graphs demonstrate that ParPIC achieves competitive clustering accuracy with improved scalability relative to spectral and teleportation-based methods.

Reinforcement Learning · Planning

Shenghua Wan, Xiaohai Hu, Xunlan Zhou, lei yuan, Le Gan, De-Chuan Zhan

The scalability of world models is currently bottlenecked by the scarcity of action annotations. While self-supervised latent action learning offers a potential solution, existing single-view paradigms—relying on information bottlenecks or Vector Quantization (VQ)—often conflate superficial 2D pixel displacements with the underlying physical-spatial dynamics of an action. Consequently, these methods remain highly susceptible to view-dependent noise, such as camera shake. We introduce **MuCoLA** (**Mu**lti-view **Co**nsistent **L**atent **A**ction learning), a framework that learns robust, view-invariant action representations by enforcing semantic consistency across synchronized video streams. MuCoLA utilizes a Student-Teacher network with DINO-style self-distillation to align action distributions across viewpoints, effectively filtering high-frequency visual noise while preserving motion semantics. Theoretical analysis reveals that our multi-view objective functions as a spectral filter, isolating agent dynamics from environmental nuisances. Empirically, MuCoLA significantly outperforms baselines in action regression, video reconstruction, and downstream visual control tasks. Furthermore, we demonstrate that MuCoLA exhibits favorable scaling properties with respect to model capacity and data volume, paving the way for large-scale action-free world modeling.

Applications · Time Series

Zipeng Wu, Jiani Wei, Shiqiao Zhou, Jiajun Chen, Fabian Spill, J. Andrews

We benchmark time series decomposition as a standalone evaluation task. While decomposition outputs are widely used to interpret trend and periodic structure, their quality is often assessed informally, and no unified benchmark exists for comparing component recovery under controlled generative mechanisms. We introduce a synthetic evaluation suite with explicit trend and cycle taxonomies, a unified interface covering representative decomposition families, and complementary metrics capturing distinct error modes (shape, phase, and spectral fidelity). Across stationary periodic regimes, STL-family methods are near-ceiling; under non-stationary periodicity (frequency drift, regime switching), fixed-period priors induce phase degradation, while subspace/time-frequency methods better preserve seasonal consistency (adaptive spectral methods may require tuning). We further extend the benchmark with a downstream scientific-discovery track---symbolic regression on decomposed components---showing that a decompose-then-regress pipeline materially improves recoverability and reduces expression complexity, linking decomposition quality to structure discovery. We release a pip-installable package and a lightweight web interface to make the benchmark and results easily accessible.