论文检索

Deep Learning · Self-Supervised Learning

Chengwei Zhou, Gourav Datta

Spiking Neural Networks (SNNs) offer a promising alternative to traditional artificial neural networks by leveraging sparse, event-driven computation that closely mimics biological neurons. When deployed on neuromorphic hardware, SNNs enable substantial energy savings due to their temporal and asynchronous processing. However, training SNNs remains fundamentally difficult because the non-differentiable nature of spike generation breaks the bidirectional gradient flow required in modern self-supervised learning (SSL) frameworks. In this work, we introduce the first fully SSL framework for SNNs that scales to large-scale visual tasks without requiring labeled fine-tuning. Our method leverages intrinsic spike-time dynamics by aligning representations across time steps and augmented views. To address gradient mismatch during surrogate training, we propose the MixedLIF neuron model, which combines a spiking path with an antiderivative-based surrogate path during training to stabilize optimization, while retaining a fully spiking and energy-efficient architecture at inference. We also introduce two temporal objectives, Cross Temporal Loss and Boundary Temporal Loss, that align multi-time-step outputs to improve learning efficiency. Our approach achieves competitive results across ResNet and Vision Transformer-based SNNs on CIFAR-10, CIFAR10-DVS, and ImageNet-1K. Our approach further generalizes through transfer learning from ImageNet-1K to downstream tasks. Notably, our self-supervised SNNs match or exceed the performance of some non-spiking SSL models, demonstrating both representational strength and energy efficiency.

Lei Lv, Yunfei Li, Yu Luo, Fuchun Sun, Xiao Ma

Iterative generative policies, such as diffusion models and flow matching, offer superior expressivity for continuous control but complicate Maximum Entropy Reinforcement Learning because their action log-densities are not directly accessible. To address this, we propose \textbf{Field Least-Energy Actor-Critic (FLAC)}, a likelihood-free framework that regulates policy stochasticity by penalizing the kinetic energy of the velocity field. Our key insight is to formulate policy optimization as a Generalized Schr\"odinger Bridge (GSB) problem relative to a high-entropy reference process (e.g., uniform). Under this view, the maximum-entropy principle emerges naturally as staying close to a high-entropy reference while optimizing return, without requiring explicit action densities. In this framework, kinetic energy serves as a physically grounded proxy for divergence from the reference: minimizing path-space energy bounds the deviation of the induced terminal action distribution. Building on this view, we derive an energy-regularized policy iteration scheme and a practical off-policy algorithm that automatically tunes the kinetic energy via a Lagrangian dual mechanism. Empirically, FLAC achieves superior or comparable performance on high-dimensional benchmarks relative to strong baselines, while avoiding explicit density estimation.

Deep Learning · Large Language Models

Haodong WANG, Junjie Liu, Zicong Hong, Qianli Liu, Jian Lin, Song Guo, Xu Chen

4-bit quantization reduces the memory footprint and latency of large language model inference, but its aggressive precision reduction can severely degrade accuracy. Prior methods address this by decomposing each weight matrix into two components (e.g., via singular value decomposition) and quantizing them separately, assigning the bulk of values to a low-precision residual component while handling outliers with a high-precision low-rank component. However, such decompositions are designed to minimize the real-valued energy of the residual, rather than the post-quantization error of the residual and low-rank components. We propose TwinQuant, a 4-bit quantization framework that learns quantization-friendly decomposed subspaces and jointly reshapes both the low-rank and residual components. TwinQuant learns component-specific transformations via a joint optimization over the Stiefel and general linear manifolds, flattening their distributions and reducing dynamic-range imbalance. To enable efficient end-to-end execution, we further design a fused dual-component kernel that pipelines the two-stage low-rank computation on-chip and merges both components with a single epilogue, avoiding intermediate global-memory traffic. Across LLaMA3 and Qwen3 models, TwinQuant preserves near-FP16 accuracy and delivers up to $2.11\times$ end-to-end speedup over an FP16 baseline.

Hao Pan, Jian Dai, Yuan Sun, Zhenwen Ren, Xingfeng Li

Multimodal image fusion integrates complementary information from different modalities. However, large cross-modal discrepancies and local conflicts often introduce uncertainty into fusion decisions. This uncertainty can bias modality allocation in inconsistent regions, leading to information loss or the propagation of artifacts. Therefore, we address this problem by formally casting image fusion as an integrated probabilistic decision system that couples prior decision-making with posterior risk minimization. Based on this view, we propose a dIfference-aware Decision-lEArning muLtimodal image fusion paradigm (IDEAL). It treats cross-modal differences as decision triggers and learns contribution policies conditioned on local conditions. Specifically, we use a difference-attention module to generate multi-scale difference maps as spatial decision conditions. We also obtain spectral conditions by projecting features into the frequency domain, where power-spectrum energy, complementary spectra, and spectral-entropy reliability characterize modality discrepancy and reliability. We then employ a symmetric Beta prior to map these decision conditions to gating weights, yielding explicit and interpretable modality contribution policies. To improve robustness, we introduce an uncertainty modulation mechanism that reverts the policy to conservative mixing when conditions is insufficient. Extensive experiments demonstrate stable and competitive performance.

General Machine Learning · Representation Learning

Juntang Wang, Hao Wu, Yihan Wang, Dongmian Zou, Shixin Xu

Clustering-based features are widely used in machine learning, but most methods must choose a resolution---a choice that is global, fixed, and ad hoc. Recent work shows that varying the resolution parameter produces only a finite set of structurally stable partitions, known as configurations. Based on this, we introduce Configuration-Mixed Prediction (CMP), a setting where models learn to adaptively weight these configurations per sample for downstream prediction. We propose MixConfig, a plug-and-play feature augmentation module that extracts configurations from any embedding and learns energy-aware mixing weights via a novel selector that jointly reasons about sample context, cluster assignments, and stability statistics. Experiments across tabular, molecular, vision, and text domains demonstrate consistent improvements over single-resolution and static baselines across diverse predictor architectures, with gains particularly pronounced in low-data regimes.

Deep Learning · Large Language Models

Senyu Han, Yilu Cao, Kai Yu, Lu Chen

Large language model (LLM) exists a subset of attention heads that are highly responsible for long-context processing. Existing work has identified different long-context heads in models, but their detection methods mainly rely on model inference on actual long texts and do not analyze the inherent properties of the head parameters. In this paper, we use kernel methods to analyze static *frequency kernels* formed by different rotation frequency components of attention heads, and we design a Long-context Potential Score (LPS) to measure the potential of attention heads in processing long contexts. Kernels of heads with high LPS exhibit concentrated low-frequency energy and low effective rank, which allow them to effectively capture highly specialized information from distant contexts. Experiments and analysis on long-context tasks and model behaviors show that the LPS metrics can well reflect the actual capability of heads on long contexts. Furthermore, by simply amplifying low-frequency kernels of heads with high retrieval potential, we can further improve model's performance on long-context tasks. Our metrics and head enhancement methods are fully static and offline, and they can be quickly conducted under low-resource constraints.

Deep Learning · Generative Models and Autoencoders

Liang Peng, Deqing Li, Yujia Wu, Hao Meng, Kuan Cao, Yu Wu, Xiaoxiao Xu, Lin Qu

Classifier-Free Guidance (CFG) is a cornerstone of flow-matching models, significantly enhancing visual quality and prompt adherence. However, high guidance scales inherently violate the optimal transport dynamics, leading to visual artifacts and mode collapse. In this paper, we investigate the mechanisms of this failure through the lens of velocity moment decomposition. Our analysis reveals that the distributional shift induced by CFG decouples into two geometric components: a Linear Barycentric Drift that shifts the global distribution center, and a Quadratic Energetic Instability that injects surplus kinetic energy, disrupting the transport cost and triggering variance explosion. To mitigate these issues, we introduce MIST (Moment-aligned Invariant Stability Transform), a training-free method designed to confine the sampling trajectory to the learned data manifold. MIST comprises two hierarchical stages: (1) Invariant Alignment (IA), a global statistical rectifier that restores structural integrity by removing the linear drift and realigning the energy profile; and (2) Stability Thresholding (ST), a local dynamical regulator that enforces Lipschitz-like smoothness via temporal decay and spatial suppression. MIST enables robust, high-fidelity generation across a wide range of guidance scales while consistently improving performance at moderate scales. Extensive experiments on diverse text-to-image and text-to-video benchmarks demonstrate that MIST outperforms standard CFG and state-of-the-art corrections, establishing a new benchmark for robust guidance in flow-based generative models.

Deep Learning · Foundation Models

Xuerui Qiu, Shaowei Gu, Peixi Wu, JiaKui Hu, Yaozhi Wen, Yuqi Pan, Xinhao Luo, Bo XU, Guoqi Li

Spiking Neural Networks (SNNs) offer an energy--efficient route to 3D spatio--temporal perception, yet they lag behind Artificial Neural Networks (ANNs) due to weak pretraining and heavy inference stacks, limiting generalization and multimodal reasoning (e.g., zero--shot 3D classification and open--world QA). We present a universal \textbf{S}pike--based \textbf{V}ision--\textbf{L}anguage pretraining framework (SVL) that equips SNNs with open--world 3D understanding while preserving end--to--end spike efficiency. SVL comprises two core components: (i) {Multi--scale Triple Alignment} (MTA), a label--free triplet contrastive objective aligning 3D, image, and text; and (ii) {Re--parameterizable Vision--Language Integration} (Rep--VLI), which converts offline text embeddings into lightweight weights for text--encoder--free inference. Moreover, we present the first fully spike--driven point Transformer, {Spike-driven PointFormer}, whose 3D spike--driven self--attention (3D-SDSA) reduces interactions to sparse additions, enabling faster, more efficient training. Extensive experiments show that SVL attains strong zero--shot 3D classification (85.4% top--1) and consistently outperforms prior SNNs on downstream tasks (e.g., +6.1% 3D cls, +2.1% DVS actions, +1.1% detection, +2.1% segmentation) while enabling open--world 3D question answering, sometimes outperforming ANNs. To the best of our knowledge, SVL represents the first scalable, generalizable, and hardware-friendly paradigm for 3D open-world understanding, effectively bridging the gap between SNNs and ANNs in complex open-world understanding tasks.

Deep Learning · Generative Models and Autoencoders

Yexin Liu, Wenjie Shu, Zile Huang, Haoze Zheng, Yueze Wang, Manyuan Zhang, Jinjing Zhu, Ser-Nam Lim, Harry Yang

Text-guided image-to-video generation has made substantial progress, yet it still struggles to execute text-specified edits that require substantial changes to a reference image (e.g., object addition, deletion, or modification). Empirically, our analysis reveals that this stems from **visual dominance**, where the reference image causes severe attention dispersion, inhibiting the model's ability to incorporate new semantic information. To address this, we propose **AlignVid**, a training-free intervention that re-calibrates the model's internal attention distribution. Drawing on an energy-based perspective of attention, AlignVid employs Attention Scaling Modulation (**ASM**) to reduce attention entropy and concentrate focus on semantic tokens, alongside Guidance Scheduling (**GS**) to maintain generation stability. To rigorously assess this capability, we present **OmitI2V**, a comprehensive benchmark for evaluating prompt adherence across object addition, deletion, and modification. Extensive experiments demonstrate that AlignVid effectively enhances semantic fidelity with negligible computational overhead.

Probabilistic Methods · Monte Carlo and Sampling Methods

James Cuin, Davide Carbone, Yanbo Tang, O. Akyildiz

The problem of optimising functions with intractable gradients frequently arise in machine learning and statistics, ranging from maximum marginal likelihood estimation procedures to fine-tuning of generative models. Stochastic approximation methods for this class of problems typically require inner sampling loops to obtain (biased) stochastic gradient estimates, which rapidly becomes computationally expensive. In this work, we develop sequential Monte Carlo (SMC) samplers for optimisation of functions with intractable gradients. Our approach replaces expensive inner sampling methods with efficient SMC approximations, which can result in significant computational gains. We establish convergence results for the basic recursions defined by our methodology which SMC samplers approximate. We demonstrate the effectiveness of our approach on the reward-tuning of energy-based models within various settings.

Reinforcement Learning · Everything Else

Fabian Wurzberger, Sebastian Gottwald, Zeqiang Zhang, Daniel A Braun

In self-supervised goal-conditioned reinforcement learning (RL) without external rewards, goals are typically specified by observations sampled from experience. However, depending on the observation structure, such a fixed representation of goals may be either too concrete (requiring exact pixel-level matches) or too abstract (involving ambiguous observations). Here we propose the construction of hierarchical latent goal spaces that integrate both concrete and abstract goals. To this end, we use an energy function to learn a partially ordered space, in which a subset relation between observations naturally induces a hierarchy from concrete to abstract goals. This representation enables agents to disambiguate specific states while also generalizing to shared concepts. In experiments on navigation and robotic manipulation, agents trained with our hierarchical goal space achieve higher task success and greater generalization to novel tasks compared to agents limited to purely observational goals.

Deep Learning · Robustness

Jiayu Xiong, Jing Wang, Qi Zhang, Wanlong Wang, Jun Xue

Real-world multimodal systems must be robust against low-quality data, such as sensor noise, incomplete multimodal data and conflicting inputs. However, existing trustworthy fusion methods rely on the model's own prediction confidence to judge data quality. This creates a circular dependency: when a model is confident but wrong (overconfident), these methods fail to detect the error. To break this loop, we propose Geometry-based Multimodal Fusion (GMF). Instead of relying on predictions, we evaluate reliability by measuring the physical effort required to map input data back to the valid data manifold. We implement this using Diffusion Schrödinger Bridges with Rectified Flow, which allows us to calculate Transport Energy as a direct metric for quality. The logic is simple: valid data sits on the manifold (low energy), while noisy, incomplete data or conflicting data requires high energy to be restored. This geometric metric acts as an independent judge, effectively flagging unreliable inputs even when the classifier is fooled. Extensive experiments demonstrate that GMF significantly improves robustness against severe sensor noise and semantic conflicts compared to confidence-based baselines.

Applications · Neuroscience, Cognitive Science

Zhenyu Liao, Di Yu, Changze Lv, Wentao Tong, Linshan Jiang, Sijie Ji, Xin Du, Hailiang Zhao, Xiaoqing Zheng, Shuiguang Deng

Millimeter-wave (mmWave) sensing enables privacy-preserving, always-on edge perception, but its measurements are often sparse, temporally irregular, and corrupted by high-frequency noise. Existing mmWave pipelines predominantly rely on artificial neural networks (ANNs), which achieve robustness through extensive preprocessing or deep architectures, thereby limiting their efficiency on edge devices. In this work, we study spiking neural networks (SNNs) for mmWave sensing from a mechanism–data alignment perspective. By leveraging the low-pass filtering behavior of leaky integrate-and-fire (LIF) dynamics, we analyze how their implicit temporal filtering interacts with the frequency structure of mmWave signals. Our analysis shows that when discriminative information resides in low-to-mid frequencies, LIF dynamics can inherently suppress high-frequency noise, clarifying when and why SNNs outperform ANNs. Based on this insight, we derive a principled criterion for configuring the membrane decay factor by matching the effective bandwidth of LIF dynamics to the data’s discriminative spectral content. Experimental results across four widely used mmWave datasets validate the proposed frequency-matching hypothesis, yielding an average test-accuracy improvement of 6.22% and a 3.64× reduction in theoretical energy consumption relative to ANN baselines, under a unified evaluation protocol.

Applications · Chemistry, Physics, and Earth Sciences

Ali Ramlaoui, Alexandre Duval, Hannah Bull, Victor Schmidt, Hugues Talbot, Fragkiskos Malliaros, Joseph Musielewicz

Machine learning interatomic potentials (MLIPs) achieve excellent accuracy when trained on large Density Functional Theory (DFT) data. To be useful in practice, they must often be adapted to target chemistries using small and expensive task-specific datasets. However, MLIPs transfer inconsistently across domains, with representations that often loose accessible composition and structure information. To address this, we present TriForces, a model-agnostic three-stream framework that separates composition and structure information, combined with self-supervised learning to preserve transferable representations. TriForces improves performance on MatBench and QM9 over baselines without needing DFT labels and enables efficient similar structure retrieval through its learned latent space. On OMat24, in limited-data training regime, TriForces reduces energy MAE by 57\% at 20K samples only and improves force MAE across sample sizes. We release pretrained TriForces variants across multiple MLIP architectures with code at https://anonymous.4open.science/r/triforces-063E.

Applications · Neuroscience, Cognitive Science

Bozhou Li, Chubo Liu, Yan Ding, Yufeng Zhang, Zhuo Tang, Kenli Li

ANN-to-SNN conversion offers energy-efficient inference but faces a fidelity-latency trade-off due to open-loop error accumulation. While conversion-aware training mitigates this, it sacrifices the generality of using off-the-shelf ANNs. We propose a closed-loop fine-tuning framework that calibrates these errors without altering the source model. Our approach employs a Dual Alignment Mechanism, utilizing global Kullback-Leibler divergence for output distillation and introducing an adaptive local Centered Kernel Alignment constraint, weighted by initial conversion loss, for feature alignment. We uncover a critical time-dependent dynamic: local constraints are essential for stabilizing representations in low-latency regimes (e.g., $T=8$) where global gradients are unstable, whereas global alignment drives fidelity at higher time steps. Experiments on CIFAR-10 demonstrate that our method achieves over 99\% of source ANN accuracy at $T=32$ (e.g., ResNet-18: 96.38\% vs.\ 96.39\%). Furthermore, this fine-tuning acts as a regularizer, yielding SNNs with input noise robustness that matches or exceeds the source ANN.

Social Aspects · Safety

Sudarshan Regmi

The ability of the deep learning model to recognize when a sample falls outside its learned distribution is critical for safe and reliable deployment. Recent state-of-the-art out-of-distribution (OOD) detection methods leverage activation shaping to improve the separation between in-distribution (ID) and OOD inputs. These approaches resort to sample-specific scaling but apply a static percentile threshold across all samples regardless of their nature. In this work, we propose AdaSCALE, an adaptive scaling procedure that dynamically adjusts the percentile threshold based on a sample's estimated OOD likelihood. This estimation leverages our key observation that OOD samples exhibit significantly more pronounced activation shifts at high-magnitude activations under minor perturbation compared to ID samples. AdaSCALE enables stronger scaling for likely ID samples and weaker scaling for likely OOD samples, creating highly separable energy scores. Our approach achieves state-of-the-art OOD detection performance, outperforming the latest rival OptFS by **14.94%** in near-OOD and **21.67%** in far-OOD datasets in average FPR@95 metric in the ImageNet-1k benchmark across eight diverse architectures.

Applications · Computer Vision

Xuanhao Qi, Tom Luan, Yukang Zhang, Jinkai Zheng, su zhou, Shuwei Li, Lei Tan

Despite significant progress in multi-modal Re-Identification (ReID), existing methods tend to emphasize low-frequency cues. Consequently, they focus on attributes such as color, illumination, and coarse appearance, while overlooking mid- and high-frequency structures that encode geometric, textural, and identity-discriminative details. This imbalance leads to incomplete spectral representations and unstable cross-modal alignment. To overcome these limitations, we introduce FUSE, a frequency-domain framework that reformulates multi-modal ReID as a two-stage process of spectral disentanglement and energy alignment. The proposed Spectral Decomposition Module (SDM) adaptively partitions features into low, mid, and high-frequency subspaces, enabling hierarchical spectral modeling. The Cross-Modal Alignment Module (CAM) further enforces energy alignment and subspace complementarity across modalities via frequency-consistency regularization. In addition, FUSE incorporates learnable frequency modulation to enhance robustness under varying illumination and heterogeneous sensor conditions. Extensive experiments on RGBNT201, RGBNT100, and MSVR310 show that FUSE achieves 9.1% mAP and 9.5% Rank-1 improvements, establishing an interpretable frequency-domain paradigm for multi-modal representation learning.

Deep Learning · Graph Neural Networks

Ruiting Zhao, Ming Li, Lixin Cui, Lu Bai, Feilong Cao, Ke Lv, Pietro Lió

Hypergraph contrastive learning is an effective paradigm for representation learning on higher-order relational data, yet existing methods largely ignore that hyperedges link nodes with multi-level semantics. Standard contrastive objectives emphasize instance discrimination via hyperspherical uniformity and tend to push embeddings apart in an indiscriminate manner. We show that this leads to a *Hierarchy–Uniformity Conflict*, whose geometric manifestation is *Semantic Flattening*, where the semantic depth of hyperedges collapses into a nearly flat cloud of instances. To address this issue, we introduce **HyperDepth**, a hypergraph contrastive learning framework that moves representations towards a hierarchy–uniformity equilibrium by jointly coordinating spectral and geometric signals. HyperDepth employs a decoupled spectral encoding scheme with adaptive gating so that high-frequency components focus on local instance discrimination while low-frequency components capture global hierarchical structure. On top of this, an energy-based hierarchical Alignment module attaches a learnable prototype tree to the representation space and minimizes an interpretable energy functional to recover the semantic depth of hyperedges. Theoretically, under a mild frequency-separation assumption, we show that the local contrastive and global hierarchical objectives operate on orthogonal spectral components and admit equilibrium embeddings that preserve semantic depth while still retaining instance-level discrimination. Experiments on 15 hypergraph datasets and 17 supervised and self-supervised baselines, spanning homophilic and heterophilic regimes, show that HyperDepth attains strong performance with the best average rank.

Ruiting Zhao, Ming Li, Lixin Cui, Lu Bai, Feilong Cao, Ke Lv, Pietro Lió

Hypergraph contrastive learning is an effective paradigm for representation learning on higher-order relational data, yet existing methods largely ignore that hyperedges link nodes with multi-level semantics. Standard contrastive objectives emphasize instance discrimination via hyperspherical uniformity and tend to push embeddings apart in an indiscriminate manner. We show that this leads to a *Hierarchy–Uniformity Conflict*, whose geometric manifestation is *Semantic Flattening*, where the semantic depth of hyperedges collapses into a nearly flat cloud of instances. To address this issue, we introduce **HyperDepth**, a hypergraph contrastive learning framework that moves representations towards a hierarchy–uniformity equilibrium by jointly coordinating spectral and geometric signals. HyperDepth employs a decoupled spectral encoding scheme with adaptive gating so that high-frequency components focus on local instance discrimination while low-frequency components capture global hierarchical structure. On top of this, an energy-based hierarchical Alignment module attaches a learnable prototype tree to the representation space and minimizes an interpretable energy functional to recover the semantic depth of hyperedges. Theoretically, under a mild frequency-separation assumption, we show that the local contrastive and global hierarchical objectives operate on orthogonal spectral components and admit equilibrium embeddings that preserve semantic depth while still retaining instance-level discrimination. Experiments on 15 hypergraph datasets and 17 supervised and self-supervised baselines, spanning homophilic and heterophilic regimes, show that HyperDepth attains strong performance with the best average rank.

Jie Jiang, Ke Cheng, Xin Xu, Mengyang Pang, Xinzhe Xu, Jiaheng Li, Yue Liu, Yuan Wang, Jun Zhang, Huan Yu 等

Generative sequence modeling faces a fundamental tension between the expressivity of Transformers and the efficiency of linear sequence models. Existing efficient architectures are theoretically bounded by shallow, single-step linear updates, while powerful iterative methods like Test-Time Training (TTT) break hardware parallelism due to state-dependent gradients. We propose PRISM (Parallel Residual Iterative Sequence Model) to resolve this tension. PRISM introduces a solver-inspired inductive bias that captures key structural properties of multi-step refinement in a parallelizable form. We employ a Write-Forget Decoupling strategy that isolates non-linearity within the injection operator. To bypass the serial dependency of explicit solvers, PRISM utilizes a two-stage proxy architecture: a short-convolution anchors the initial residual using local history energy, while a learned predictor estimates the refinement updates directly from the input. This design distills structural patterns associated with iterative correction into a parallelizable feedforward operator. Theoretically, we prove that this formulation achieves Rank-$L$ accumulation, structurally expanding the update manifold beyond the single-step Rank-$1$ bottleneck. Empirically, it achieves comparable performance to explicit optimization methods while achieving \textbf{174x higher throughput}. Codes are available in \href{https://anonymous.4open.science/r/msir-F607/}{anonymous.4open.science}.