Dataset distillation or condensation aims to condense a large-scale training dataset into a much smaller synthetic one such that the training performance of distilled and original sets on neural networks are similar. Although the number of training samples can be reduced substantially, current state-of-the-art methods heavily rely on enormous soft labels to achieve satisfactory performance. As a result, the required storage can be comparable even to original datasets, especially for large-scale ones. To solve this problem, instead of storing these heavy labels, we propose a novel label-lightening framework termed HeLlO aiming at effective image-to-label projectors, with which synthetic labels can be directly generated online from synthetic images. Specifically, to construct such projectors, we leverage prior knowledge in open-source foundation models, e.g., CLIP, and introduce a LoRA-like fine-tuning strategy to mitigate the gap between pre-trained and target distributions, so that original models for soft-label generation can be distilled into a group of low-rank matrices. Moreover, an effective image optimization method is proposed to further mitigate the potential error between the original and distilled label generators. Extensive experiments show that our method significantly reduces the storage cost to merely 0.001% compared to full soft-label storage methods while achieving comparable performance to state-of-the-art dataset distillation methods on large-scale datasets. Our codes are available at https://github.com/Lexie-YU/HeLlO.
论文检索
输入标题、作者或关键词，从 2,893 篇学术成果中精准定位
Optimal Transport for Brain-Image Alignment: Unveiling Redundancy and Synergy in Neural Information Processing
PDF ↗The design of artificial neural networks (ANNs) is inspired by the structure of the human brain, and in turn, ANNs offer a potential means to interpret and understand brain signals. Existing methods primarily align brain signals with stimulus signals using Mean Squared Error (MSE), which focuses only on local point-wise alignment and ignores global matching, leading to coarse interpretations and inaccuracies in brain signal decoding. In this paper, we address these issues through optimal transport (OT) and theoretically demonstrate why OT provides a more effective alignment strategy than MSE. Specifically, we construct a transport plan between brain voxel embeddings and image embeddings, enabling more precise matching. By controlling the amount of transport, we mitigate the influence of redundant information. We apply our alignment model directly to the Brain Captioning task by feeding brain signals into a large language model (LLM) instead of images. Our approach achieves state-of-the-art performance across ten evaluation metrics, surpassing the previous best method by an average of 6.11% in singlesubject training and 3.81% in cross-subject training. Additionally, we have uncovered several insightful conclusions that align with existing brain research. We unveil the redundancy and synergy of brain information processing through region masking and data dimensionality reduction visualization experiments. We believe our approach paves the way for a more precise understanding of brain signals in the future. The code is available at https://github.com/NKUShaw/OT-Alignment4brain-to-image
In this paper, we propose Binarized Change Detection (BiCD), the first binary neural network (BNN) designed specifically for change detection. Conventional network binarization approaches, which directly quantize both weights and activations in change detection models, severely limit the network's ability to represent input data and distinguish between changed and unchanged regions. This results in significantly lower detection accuracy compared to real-valued networks. To overcome these challenges, BiCD enhances both the representational power and feature separability of BNNs, improving detection performance. Specifically, we introduce an auxiliary objective based on the Information Bottleneck (IB) principle, guiding the encoder to retain essential input information while promoting better feature discrimination. Since directly computing mutual information under the IB principle is intractable, we design a compact, learnable auxiliary module as an approximation target, leading to a simple yet effective optimization strategy that minimizes both reconstruction loss and standard change detection loss.Extensive experiments on street-view and remote sensing datasets demonstrate that BiCD establishes a new benchmark for BNN-based change detection, achieving state-of-the-art performance in this domain.
With the rapid development of wireless communication technology, the efficient utilization of spectrum resources, optimization of communication quality, and intelligent communication have become critical. Radio map reconstruction is essential for enabling advanced applications, yet challenges such as complex signal propagation and sparse observational data hinder accurate reconstruction in practical scenarios. Existing methods often fail to align physical constraints with data-driven features, particularly under sparse measurement conditions. To address these issues, we propose Physics-Aligned Radio Map Diffusion Model (PhyRMDM), a novel framework that establishes cross-domain representation alignment between physical principles and neural network features through dual learning pathways. The proposed model integrates Physics-Informed Neural Networks (PINNs) with a representation alignment mechanism that explicitly enforces consistency between Helmholtz equation constraints and environmental propagation patterns. Our architecture employs two synergistic U-Nets: the first ensures physical consistency by minimizing PDE residuals and boundary conditions through latent space alignment, while the second refines predictions via diffusion-based denoising with attention-guided feature fusion. This dual alignment strategy enables simultaneous satisfaction of wave propagation laws and data distribution characteristics. Experimental results demonstrate significant improvements over state-of-the-art methods, achieving NMSE of 0.0031 and RMSE of 0.0125 under Static Radio Map (SRM) conditions, and NMSE of 0.0047 with RMSE of 0.0146 in Dynamic Radio Map (DRM) scenarios. The proposed representation alignment paradigm provides 37.2% accuracy enhancement in ultra-sparse cases (1% sampling rate), confirming its effectiveness in bridging physics-based modeling and deep learning for radio map reconstruction. These advancements establish a new framework for sparse signal environment characterization, with direct applications in 5G/6G network optimization and intelligent spectrum management. The code can be found on the website: https://github.com/Hxxxz0/RMDM
Video analytics pipelines migrating to edge deployments are facing performance bottlenecks under limited bandwidth. Non-uniform intra-frame encoding emerges to further compress pixels without affecting the output of the server deep neural network (DNN), while it is inefficient in high-resolution video streaming at low bandwidth. The detail enhancement capability of neural super-resolution (SR) permits resolution downsampling and aggressive compression on edge devices for low-latency transmission. To exploit its accuracy potential, DNN-oriented non-uniform encoding is expected to be additionally aware of SR models. However, traditional codecs struggle to cope with both quality optimization for SR and global semantic features for DNN. We advocate neural codecs for coordinated encoding and enhancement, enabling analytic-oriented video streaming with optimal accuracy-delay tradeoffs. Our system, VidIQ, achieves quality-enhanced real-time video analytics by 1) improving the network architecture of neural codecs (at two granularity) to integrate SR models into a DNN-oriented analytics pipeline, and 2) adapting the multi-scale encoder and SR-decoder to scene dynamics (i.e., content and bandwidth variations) with the help of the monolithic controller to hold a performance advantage. Extensive evaluations showcase that VidIQ reduces end-to-end delay by 35.8% and improves analytics accuracy by 21.2% compared to the recent video compression, enhancement, and streaming baselines.
Federated learning (FL), an emerging data-secure distributed training paradigm, unites massive isolated Internet of Things (IoT) device nodes to collaboratively train a global neural network (NN) model without the exposure of their local multimedia data. However, constrained by the synchronous NN model integration nature of FL, there is a training latency inconsistency among heterogeneous devices, which significantly deteriorates FL training efficiency. Meanwhile, frequent local NN training and transmission impose high energy consumption pressure on users. To tackle these issues, this paper proposes a premium multi-width NN-assisted hierarchical FL (HFL) framework in heterogeneous cloud-edge-device computing to achieve remarkable training speedup and energy conservation. Specifically, a heterogeneity-aware NN width coefficient determination algorithm, which flexibly assigns a subnet with a suitable width to each user device based on its computing ability, is first applied to shorten the HFL training latency. Subsequently, to integrate subnets with different width topologies, we design a width-aware adaptive NN model integration approach to effectively ensure the accuracy of the integrated global NN model. Finally, a latency-aware energy saving strategy is introduced to reduce energy consumption. Experimental results demonstrate that our proposed framework outperforms state-of-the-art benchmarks, and attains up to 42.42% enhancement in accuracy, 81.5% reduction in training latency, and 40.9% optimization in energy cost.
As the development of lightweight deep learning algorithms, various deep neural network (DNN) models have been proposed for the remote sensing scene classification (RSSC) application. However, it is still challenging for these RSSC models to achieve optimal performance among model accuracy, inference latency, and energy consumption on resource-constrained edge devices. In this paper, we propose a lightweight RSSC framework, which includes a distilled global filter network (GFNet) model and an early-exit mechanism designed for edge devices to achieve state-of-the-art performance. Specifically, we first apply frequency domain distillation on the GFNet model to reduce model size. Then we design a dynamic early-exit model tailored for DNN models on edge devices to further improve model inference efficiency. We evaluate our E3C model on three edge devices across four datasets. Extensive experimental results show that it achieves an average of 1.3x speedup on model inference and over 40% improvement on energy efficiency, while maintaining high classification accuracy.
Multivariate time-series (MTS) classification tasks play a key role in data-driven applications spanning healthcare, finance, and mobile communication. As MTS data are typically collected from multiple interdependent sensors, the resulting temporal patterns inherently reflect the characteristics of the underlying sensing systems. Despite this connection, conventional MTS classification models predominantly focus on raw time-series data while disregarding valuable sensor-specific prior knowledge, which fundamentally constrains their classification accuracy. The emergence of large language models (LLMs) has encoded extensive sensor-related knowledge within their parameter spaces. However, effectively harnessing such knowledge to enhance MTS classification networks remains an open challenge. To address this, we propose Foresail, a status-guided neural framework that bridges this gap through systematic integration of LLM-derived sensor knowledge via the status relationship matrix and fine-grained status labels. Foresail can be seamlessly integrated with existing MTS networks to optimize performance and generate interpretable intermediate results. Experiments on irregularly and regularly sampled MTS data demonstrate that Foresail outperforms state-of-the-art approaches, achieving a notable improvement in F1-score of up to 10.9% compared to the basic MTS network.
Multimodal spiking neural networks (SNNs) hold significant potential for energy-efficient sensory processing but face critical challenges in modality imbalance and temporal misalignment. Current approaches suffer from uncoordinated convergence speeds across modalities and static fusion mechanisms that ignore time-varying cross-modal interactions. We propose the temporal attention-guided adaptive fusion framework for multimodal SNNs with two synergistic innovations: 1) The Temporal Attention-guided Adaptive Fusion (TAAF) module that dynamically assigns importance scores to fused spiking features at each timestep, enabling hierarchical integration of temporally heterogeneous spike-based features; 2) The temporal adaptive balanced fusion loss that modulates learning rates per modality based on the above attention scores, preventing dominant modalities from monopolizing optimization. The proposed framework implements adaptive fusion, especially in the temporal dimension, and alleviates the modality imbalance during multimodal learning, mimicking cortical multisensory integration principles. Evaluations on CREMA-D, AVE, and EAD datasets demonstrate state-of-the-art performance (77.55%, 70.65% and 98.65% accuracy, respectively) with energy efficiency. The system resolves temporal misalignment through learnable time-warping operations and faster modality convergence coordination than baseline SNNs. This work establishes a new paradigm for temporally coherent multimodal learning in neuromorphic systems, bridging the gap between biological sensory processing and efficient machine intelligence.mfp
How far are we really from automatically generating neural networks? While neural network weight generation shows promise, current approaches struggle with generalization to unseen tasks and practical application exploration. To address this, we propose T2W, a diffusion transformer framework that generates task-specific weights conditioned on natural language descriptions. T2W hierarchically processes network parameters into uniform blocks, integrates text embeddings from CLIP via a prior attention mechanism, and employs adversarial training with weight-space augmentation to enhance generalization. Experiments on Cifar100, Caltech256, and TinyImageNet demonstrate T2W's ability to produce high-quality weights for unseen tasks, outperforming optimization-based initialization and enabling novel applications such as weight enhancement and text-guided model fusion. Our work bridges textual semantics with weight-space dynamics, supported by an open-source dataset of text-weight pairs, advancing the practicality of generative models in neural network parameter synthesis. Our code is available on https://github.com/TianSuya/T2W.
This paper studies the problem of graph out-of-distribution generalization, which aims to enhance the performance of graph neural networks (GNNs) under distribution shifts. Existing approaches usually learn graph representations from a casual graph, which may not explicitly utilize environment information explicitly. Furthermore, they could suffer from performance degradation when confusing semantics related to target labels and environments. In this paper, we propose a novel approach named Dual Prompt Learning with Information Bottleneck (DATE) for graph out-of-distribution generalization. The core of our DATE is to utilize dual prompts to extract task-oriented semantics and model distribution shifts, respectively. In particular, we first pre-train a GNN using contrastive learning with pretext tokens introduced. More importantly, we not only introduce a task-oriented prompt based on LLMs to generate environment-invariant representations, but also learn the environment-oriented prompts to simulate subgraphs in different environments. To optimize our prompts, we introduce a graph information bottleneck framework, which minimizes the mutual information between environment-invariant representations and environment semantics with the most semantics preserved. Extensive experiments on various benchmark datasets validate the effectiveness of our DATE against various state-of-the-art approaches.
Screenshot tools bring convenience to daily workflows but simultaneously pose risks of screen content leakage. Most existing image watermarking methods struggle to protect dynamic screen content (DSC) due to two key limitations: low generalizability across diverse file types and reliance on cover images, which cause difficulties in handling diverse and dynamic screen content. Recently proposed screen-targeted watermarking methods offer cover-independent, fast-response solutions for DSC protection, but they mainly support large-scale screenshots and struggle to balance robustness and visual quality, limiting real-world applicability. To address these issues, in this work, we propose DynMark, a novel cover-independent watermarking scheme for DSC protection. Our method generates a watermark mask that is directly overlaid onto the screen surface for embedding, without relying on the underlying content. As a result, the approach maintains the same watermark mask even when the screen content changes, ensuring stability without the need for updates. Specifically, we use an invertible neural network (INN) to generate watermark and location blocks, jointly optimized with the decoder and locator. Additionally, edge smoothing is applied to further enhance visual quality. These components are integrated into a three-stage training framework to ensure robust performance. This design ensures stable extraction even from small screenshots with size down to 256 x 256, overcoming the limitations of existing methods regarding screenshot size. Extensive experiments show that our method achieves superior visual quality, extraction accuracy, and adaptability to different screenshot tools and screen resolutions, offering an efficient and practical solution for protecting screen content.
Localizing text under low-light conditions has gained attention, with typical approaches relying on two stage cascading modules that combine low-light enhancement and text localization. However, these often require additional enhancement modules and cause inefficiency in joint optimization. In this work, we address the challenge by adopting a novel approach: tailoring the detector for low light conditions through knowledge distillation from normal light conditions, without relying on any enhancement module. First, we design a Graph Topological Aggregation (GTA) model that utilizes the message passing mechanism of graph neural networks to structurally represent text topology and facilitate structured feature expression in knowledge transfer. We then introduce two specially designed knowledge transfer constraints aimed at enhancing the learning of text's multi-scale features and topological knowledge. Finally,we propose a Stage-wise Bilevel Knowledge Transfer learning strategy that designates the low-light learning process as the upper-level task, while treating normal light learning as the lower-level task, effectively addressing the coupling issues and sequential dependencies prevalent during the distillation process. Extensive experiments underscore the approach's superiority.
Deep neural networks (DNNs) are susceptible to Universal Adversarial Perturbation (UAP), which significantly increases the likelihood of deceiving DNNs. Current UAP generation methods are categorized into data-dependent, relaxed-data-free, and data-free attacks based on different data dependencies. However, both strategies exhibit poor transferability in the black-box settings. To address this limitation, we propose BTUAP, a novel UAP generation method designed to enhance the transferability of UAP in the black-box setting. BTUAP employs an ensemble strategy with min-max weight adjustment mechanisms to reduce the impact of model characteristics and introduces a self-supervised optimization strategy to maximize the distance of predicted logits between benign samples and adversarial examples. Experimental results demonstrate that BTUAP significantly improves transferability in different data dependency settings under black-box constraints. We also quantify the impact of the distribution shift and provide a new metric to measure the robustness of models. The source code is available at htps://github.com/RaymondDawn/BTUAP.
Deep Neural Networks (DNNs) have become increasingly prevalent in various applications, yet they remain vulnerable to adversarial attacks, particularly through the use of adversarial examples (AEs). This paper introduces the concept of stealthy AE, which is benign before transmission through Online Social Networks (OSNs) but becomes adversarial after processing. The inherent transformations applied by OSNs, such as image compression and format conversion, can activate the properties of adversarial examples that are originally hidden. We present a suite of stealthy AE generation frameworks. Subsequently, our scheme involves the quality factor calculation, leveraging the diffusion model with differential JPEG layers to simulate OSN transmission, and utilizing the Lagrange multiplier method for AE generation optimization. Extensive experiments demonstrate that our method consistently outperforms seven state-of-the-art adversarial example generation techniques across multiple OSNs and victim models. Moreover, resistance detection evaluation and extended experiments with different attack settings also demonstrated the scalability of our scheme.
Spiking Neural Network (SNN), as a next-generation neural network technology, use binary spike signals as carriers of information. They offer advantages such as low energy consumption, low computational complexity, and high information transmission rates. However, deep SNNs suffer from the gradient vanishing problem due to issues like the non-differentiability of the step function and neuron dormancy. To address this gradient problem, we first propose the MCLIF neuron, which optimizes the backpropagation mechanism and compensates for gradient information from both temporal and spatial dimensions. Furthermore, we design a spiking attention mechanism tailored to the temporal characteristics of SNNs. By introducing the QK Memory to embed temporal features, we make full use of information from different time steps. Additionally, we propose a gradient correction module to enhance the model's representational power from both temporal and spatial dimensions. The proposed SGM-Transformer achieves state-of-the-art (SOTA) performance in image classification tasks such as CIFAR10, CIFAR100, and CIFAR10-DVS, and also excels in industrial defect classification scenarios. The code will be made available after the paper is accepted.
The natural combination of intricate topological structures and rich textual information in text-attributed graphs (TAGs) opens up a novel perspective for graph anomaly detection (GAD). However, existing GAD methods primarily focus on designing complex optimization objectives within the graph domain, overlooking the complementary value of the textual modality, whose features are often encoded by shallow embedding techniques, such as bag-of-words or skip-gram, so that semantic context related to anomalies may be missed. To unleash the enormous potential of textual modality, large language models (LLMs) have emerged as promising alternatives due to their strong semantic understanding and reasoning capabilities. Nevertheless, their application to TAG anomaly detection remains nascent, and they struggle to encode high-order structural information inherent in graphs due to input length constraints. For high-quality anomaly detection in TAGs, we propose CoLL, a novel framework that combines LLMs and graph neural networks (GNNs) to leverage their complementary strengths. CoLL employs multi-LLM collaboration for evidence-augmented generation to capture anomaly-relevant contexts while delivering human-readable rationales for detected anomalies. Moreover, CoLL integrates a GNN equipped with a gating mechanism to adaptively fuse textual features with evidence while preserving high-order topological information. Extensive experiments demonstrate the superiority of CoLL, achieving an average improvement of 13.37% in AP. This study opens a new avenue for incorporating LLMs in advancing GAD.
Multi-view clustering plays a pivotal role in remote sensing image analysis, where graph neural network-based methods have demonstrated remarkable potential by modeling data as graphs. However, existing efforts, which construct remote sensing graphs using fixed rules (e.g., K-nearest neighbors), inevitably introduce noisy edges and increase the risk of heterogeneous information diffusion, leading to inferior clustering performance. Although recent works attempt to address this issue by refining the structure, they are designed for single-view data and struggle to extend to multi-view scenarios. To bridge this gap, we propose a dual structure awareness multi-view graph clustering method named DSMVGC, which generates two distinct structures for each view through explicit and implicit perspectives. Specifically, in our method, the learning processes of structure refinement and clustering are alternately optimized to mutually enhance each other. On one hand, the explicit structure updates the topology based on inter-cluster relationships, while the implicit structure captures latent relationships not covered by the explicit structure through adversarial learning. On the other hand, the refined structures not only facilitate homogeneous message passing but also serve as prior knowledge to guide the contrastive loss, thereby enhancing the discriminability of representations for accurate clustering. Extensive experiments on five multi-view remote sensing datasets validate the effectiveness of DSMVGC.
Drug-drug interaction (DDI) prediction is a pivotal task in biomedical research. Emerging multimodal approaches that integrate graph neural networks (GNNs) and large language models (LLMs) have gained traction, as GNNs capture molecular structures while LLMs provide a rich biomedical context. However, real-world DDI data often exhibit distribution shifts across structural and textual dimensions, stemming from variations in molecular scaffolds, drug sizes, and assay conditions. Existing methods assume an independent and identically distributed (I.I.D.) setting, failing to handle such shifts primarily due to there key limitations: (i) the entanglement of core interaction motifs with incidental structural features; (ii) inflexible message-passing GNN architectures ill-suited for diverse drug pairs; and (iii) underutilized biomedical knowledge in LLMs for capturing pairwise interaction semantics. These limitations highlight the need for a disentangled, dynamic, and pairwise-aware modeling strategy to achieve out-of-distribution generalized DDI prediction. To solve this problem, we propose DyNamic Pairwise Architecture Search for Generalizable Drug-Drug Interaction LLM (DyNAS-DDI), a novel framework that dynamically adapts network architectures for each molecular pair and integrates biomedical knowledge from LLMs to improve generalization under distribution shifts. Specifically, we propose three modules: (i) Motif-driven disentangled molecule encoding, which disentangles molecular representations into distinct motif-based features while preserving key structural signals through a self-supervised graph encoder; (ii) Attentionbased pairwise neural architecture search, where multi-head attention enriches molecular features to guide a dynamic search mechanism that adaptively optimizes message passing for diverse interaction types; and (iii) retrieval-augmented molecular instruction tuning, where external biomedical knowledge is incorporated to improve interpretability and enable reasoning for unseen drug interactions. Extensive experiments on four datasets for DDI with out-of-distribution (OOD) splits demonstrate our method's superior generalization abilities under distribution shifts. Our code can be available at https://github.com/EkkoXiao/DyNAS-DDI.
Deep multi-view graph clustering seeks to integrate diverse graph feature sets and uncover consistent information across multiple views. While extensive prior research has utilized various neural network architectures to address multi-view graph clustering challenges, these approaches exhibit notable limitations: 1) The ''black-box'' nature of deep learning models, which obscures their internal mechanisms and impedes interpretability; 2) Insufficient efforts aim to capture low-dimensional representations through graphs that reflect intuitive clustering structures and reduce computational cost. To address these limitations, this paper introduces an interpretable multi-view graph clustering framework constructed with optimization-inspired modules. The proposed approach formulates low-dimensional clustering representation learning from graph matrices as an optimization problem, deriving an iterative solution rooted in this formulation. By seamlessly bridging this optimization process to a deep network architecture, the model learns a low-dimensional clustering representation for graph-structured data across multiple views while adhering to the iterative optimization principles and reducing computational costs. This transparent network design enhances the interpretability of multi-view clustering, enabling intuitive and human-understandable learning of clustering structures. Extensive experimental evaluations validate the proposed framework's superiority over state-of-the-art methods in multi-view clustering tasks while ensuring interpretability and reducing computational costs.