Class-Incremental Learning (CIL) aims to enable models to sequentially learn new tasks while retaining knowledge from previous ones. Recently, merging-based pre-trained CIL methods have gained significant attention due to their competitive performance and high inference efficiency. However, most existing approaches decouple training from merging, neglecting the compatibility among task-specific adapters. This incompatibility introduces severe conflicts during integration, resulting in catastrophic forgetting and notable performance degradation. To address this limitation, we propose MOBO, a Merging-Oriented Bi-Level Optimization framework that synergizes the optimization of the current task model with the performance of the final merged model. At the upper level, we introduce a global loss that anticipates the merged model's behavior, guiding the current task parameters toward a solution space that facilitates effective merging. At the lower level, we employ a dynamic weighted merging strategy to optimize merging coefficients and update the merged model. By alternately optimizing the task-specific and merged models, MOBO effectively mitigates the performance loss caused by incompatible adapter integration. Comprehensive experiments on CIFAR-100, CUB-200, ImageNet-R, ImageNet-A, and VTAB demonstrate the superiority of our approach, highlighting its robustness and scalability, particularly on long task sequences.
论文检索
输入标题、作者或关键词，从 7,876 篇学术成果中精准定位
Detecting text generated by large language models~(LLMs) is crucial but challenging. Existing detectors depend on impractical assumptions, such as white-box settings, or solely rely on text-level features, leading to imprecise detection ability. In this paper, we propose a simple but effective and practical LLM-generated text detection method, VaryBalance. The core of VaryBalance is that, compared to LLM-generated texts, there is a greater difference between human texts and their rewritten version via LLMs. Leveraging this observation, VaryBalance quantifies this through mean standard deviation and distinguishes human texts and LLM-generated texts. Comprehensive experiments demonstrated that VaryBalance outperforms the state-of-the-art detectors, i.e., Binoculars, by up to 34.3% in terms of AUROC, and maintains robustness against multiple generating models and languages.
Vision Transformers (ViTs) exhibit notable susceptibility to adversarial attacks, presenting a significant challenge for their deployment in security-sensitive applications. Despite their considerable empirical successes, a rigorous theoretical foundation for ViT's adversarial generalization behavior has not been adequately established. To address this limitation, we leverage empirical Rademacher complexity to analyze the mechanism of perturbation accumulation through deep ViTs layers. We establish a high-probability generalization bound for ViTs in classification tasks under adversarial settings. Our theoretical framework elucidates the roles of several factors in mitigating perturbation effects, norm regularization of weight matrices (in both MLP and attention modules) and depth-wise propagation constraints on layer-wise norms. Extensive experiments on benchmark datasets corroborate our theoretical insights, bridging the gap between ViTs architecture design and adversarial robustness.
Graph Masked AutoEncoders (GMAEs) have achieved notable success in diverse downstream tasks. However, their superior performance usually relies on reliable and unperturbed input graphs. In real-world scenarios, graphs are vulnerable to adversarial attacks (e.g., perturbations on node features or topology). As a result, existing GMAEs tend to overfit these perturbed inputs, leading to poor robustness. Motivated by this, we propose an adversarial masked graph modeling for robust graph autoencoders, named ArmorGAE. Specifically, we first design a dual-consistency graph augmentation method that enriches the graph topology to mitigate structural vulnerabilities. Theoretically, we establish a connection between ArmorGAE and contrastive learning. It demonstrates that our augmentation strategy in GMAEs is equivalent to effectively expanding the set of high-quality positive samples in contrastive learning, providing richer and more reliable supervisory signals. We then present an adversarial masked autoencoder to mitigate model overfitting to the perturbed graph. The generator and discriminator are jointly trained under an adversarial paradigm, where the former aims to recover the underlying true data distribution, while the latter distinguishes reconstructed and real edges. Experimental results on five datasets demonstrate that ArmorGAE outperforms state-of-the-art methods in terms of classification accuracy on clean graphs and adversarial robustness under 16 distinct adversarial scenarios. The codes are publicly available at https://github.com/ZZY-GraphMiningLab/ArmorGAE.
Graph self-supervised learning aims to mine intrinsic signals from graph data itself to train models. It enables the acquisition of high-quality representations without manual annotations, making it suitable for various label-scarce scenarios and thus garnering substantial interest. Existing graph self-supervised methods typically assume that each node possesses a single semantic meaning. However, many real-world entities exhibit multiple semantics, where a single node may simultaneously belong to multiple categories. Since existing approaches are mainly designed for single-semantic settings, they often struggle to accurately capture the multiple semantics within nodes. Meanwhile, existing multi-label graph learning methods mainly depend on extensive manual annotations, which are costly and of limited applicability. To address this, we make a bold attempt to extend graph self-supervised learning to multi-label graphs. It is particularly challenging, as lacking labels makes it difficult to identify multiple semantics, let alone determine the number of underlying categories for nodes. To this end, we propose a Multi-semantic Aware Self-Supervised pretraining method (MASS) for multi-label graphs. Specifically, we propose a multi-pseudo-label decomposition mechanism, enabling adaptive learning of multiple semantics within nodes without annotations. Extensive results validates the effectiveness of MASS, and it achieved competitive performances with supervised baselines.
Existing AI-generated image (AIGI) detectors perform well in-domain but degrade severely under distribution shift. We observe that this failure is mainly caused by content shortcuts, where detectors spuriously couple forgery artifacts with semantic content, such as object categories or demographic attributes, learning content–label correlations instead of generalizable forgery patterns. To address this issue, we propose PURE (Purging Unrelated Representations for Content-Agnostic Forgery Detection), which achieves content-agnostic detection through two complementary components: a Causal Semantic Generative (CSG) mechanism that disentangles semantic representations from forgery-irrelevant nuisance factors, and a Gaussian Mixture Model (GMM)-based prototype alignment module that suppresses category-specific content bias. Extensive experiments on CIFAKE, GenImage, and AlFace show that PURE achieves superior generalization under spurious correlation reversal. The code is available at https://github.com/wuxinyu519/PURE.
Coverage-based prime implicant explanations are formal explanations offering a number of valuable assets, especially in terms of faithfulness and generality. Unfortunately, deriving a coverage-based prime implicant explanation for an instance is computationally hard in the general case (the problem of identifying such an explanation being at the second level of the polynomial hierarchy). In this paper, we focus on the computation of a coverage-based prime implicant explanation for an instance given a tree-based model. We show that the specific nature of the domain theory linking the Boolean conditions in such models makes the problem computationally easier. We present a greedy algorithm to derive coverage-based prime implicant explanations when dealing with a tree-based model. We also present an empirical evaluation showing that this algorithm is efficient enough to be used in many practical cases.
Multivariate Time Series Classification (MTSC) demands models that can effectively capture complex temporal patterns across multiple scales while remaining computationally efficient. However, existing approaches generally struggle to reconcile fine-grained representation learning, especially under class imbalance and real-world constraints. In this paper, we present FreSH, a Frequency-Segmented Hierarchical Multi-Expert Framework designed to address these challenges. FreSH introduces a new perspective for MTSC by enabling adaptive, multi-scale analysis of temporal signals, allowing different aspects of the data to be modeled in a complementary and coordinated manner. By combining localized specialization with holistic context modeling, FreSH achieves strong representational capacity without incurring excessive computational overhead. An adaptive fusion strategy further enhances flexibility, enabling the model to dynamically emphasize the most informative components of the input. In addition, we incorporate a more robust optimization objective that improves learning stability across varying sample difficulties and class distributions. Extensive evaluations on 30 UEA benchmark datasets and real-world vibration data demonstrate that FreSH consistently outperforms state-of-the-art methods in classification accuracy, while substantially reducing model size and efficiency. The implementation code is publicly available at https://github.com/Wangmy2120/FreSH00.
Federated Learning (FL) enables distributed learning across multiple clients without sharing raw data. When statistical heterogeneity across clients is severe, Clustered Federated Learning (CFL) can improve performance by grouping similar clients and training cluster-wise models. However, most CFL approaches rely on multiple communication rounds for cluster estimation and model updates, which limits their practicality under tight constraints on communication rounds. We propose Data Collaboration-based Clustered Federated Learning (DC-CFL), a single-round framework that completes both client clustering and cluster-wise learning, using only the information shared in DC analysis. DC-CFL quantifies inter-client similarity via total variation distance between label distributions, estimates clusters using hierarchical clustering, and performs cluster-wise learning via DC analysis. Experiments on multiple open datasets under representative non-IID conditions show that DC-CFL achieves accuracy comparable to multi-round baselines while requiring only one communication round. These results indicate that DC-CFL is a practical alternative for collaborative AI model development when multiple communication rounds are impractical. Our source code is publicly available at https://github.com/soutasuga/DC-CFL.
Graph Neural Networks (GNNs) are vulnerable to backdoor attacks, where models behave normally on clean data but exhibit targeted misclassifications once specific triggers are activated. Existing backdoor attacks on GNNs mainly focus on enhancing trigger stealthiness or diversifying attack paradigms. However, these methods overlook a fundamental property of GNNs: trigger-induced malicious signals inevitably propagate through graph neighborhoods, causing unintended mispredictions on clean nodes, i.e., Collateral Damage. To address this issue, we propose the Collateral Damage Constrained Graph Backdoor Attack (CDCA), a novel framework that explicitly controls malicious diffusion. Specifically, the proposed method combines neighborhood-aware target node selection with a self-constrained trigger generation strategy to suppress trigger-induced propagation by enforcing prediction consistency on clean K-hop neighboring nodes. Extensive experiments on real-world datasets demonstrate that the proposed method remains effective while significantly reducing collateral damage.
Continual adversarial defense (CAD) aims to defend target models against continuously emerging attacks. However, existing CAD methods typically rely on maintaining large amounts of replay data or multiple expert modules to mitigate catastrophic forgetting, or suffer from reduced robustness against adversarial examples or degraded performance on clean images. As a result, they often fail to provide clear advantages over standalone robust models. To address these limitations, we formulate four fundamental principles for CAD: (1) continual adaptation to new attacks without catastrophic forgetting, (2) few-shot adaptation, (3) memory-efficient adaptation, and (4) high classification accuracy on both clean and adversarial data. Guided by these principles, we explore and integrate cutting-edge techniques from continual learning, few-shot learning, and ensemble learning, and propose Universal Continual Adversarial Defense (UCAD), a universal framework that enables both standard and robust models to perform effective defense under the CAD setting. Extensive experiments validate the effectiveness of UCAD against multi-stage adversarial attacks and demonstrate significant improvements over a wide range of baseline methods. Moreover, we observe that as the number of encountered attacks increases, UCAD becomes increasingly robust, consistently enhancing the defense capability of existing robust models until saturation.
Graph pre-training and prompt tuning provide an effective route to label-efficient node classification by learning transferable backbones and adapting them with lightweight prompts. However, existing pre-train-and-prompt pipelines often generalize poorly across graphs with diverse homophily due to two fundamental limitations: (i) spectral bias, where learned representations overemphasize a single frequency band, and (ii) prompt misalignment, where spatial-domain prompts cannot explicitly modulate critical spectral components and may even disrupt frequency-aware backbones. In this paper, we propose SCA-GPPT, a spectrum-aware framework that unifies Spectrum-based Graph Pre-training and Cluster-Augmented Prompt Tuning. First, we present a systematic spectral analysis of graph prompting, revealing the inherent limitations of spatial-domain prompts in capturing attenuated high-frequency signals. Second, we explicitly learn complementary low- and high-frequency spectral filters via parameterized Chebyshev polynomials and employ band-aware contrastive learning to model diverse spectral patterns during pre-training. For downstream adaptation, we freeze the pre-trained backbone and introduce cluster-augmented prompts that directly adapt Chebyshev coefficients while enforcing global structural consistency, enabling stable few-shot transfer across varying homophily regimes. Extensive experiments on real-world benchmarks demonstrate that SCA-GPPT consistently outperforms strong baselines under both transductive and inductive settings. Our code is available at https://anonymous.4open.science/r/SCA-GPPT-17B2.
Deep learning has achieved remarkable success in automated electrocardiogram (ECG) diagnosis. However, the long-tailed distribution of real-world ECG data remains a critical challenge. Existing methods mainly operate at the signal feature level and fail to exploit semantic relationships among diagnostic labels, which limits their ability to effectively recognize rare categories. To address this issue, we propose a Semantics-Guided Representation Learning (SGRL) framework, designed to leverage semantic priors to guide feature learning and thereby correct feature bias under long-tailed distributions. Specifically, we propose a Label Semantic Prompt Adaptation (LSPA) strategy, which introduces ECG-calibrated learnable parameters into label prompts to generate adaptive label semantic embeddings under a semantic-consistency constraint, thereby overcoming the limitations of coarse-grained label prompts in characterizing fine-grained intra-class morphological variations. Subsequently, we propose Semantic-Guided Representation Decorrelation (SGRD) that fuses ECG embeddings with adaptive label semantics and suppresses inter-class semantic correlations to obtain discriminative class semantic anchors, thereby improving tail-class separability. Extensive experiments on multiple public datasets demonstrate the significant superiority of SGRL in long-tailed ECG classification tasks. Our code is available on https://github.com/gfywudi/SGRL.
Conventional threat detection methodologies encounter substantial limitations in real-world complex network settings, owing to their dependence on single-dimensional analysis, vulnerability to adversarial perturbations, and propensity for overfitting. This study introduces an adversarial training optimization framework that incorporates hierarchical label encoding and prompt learning, designed to enhance model robustness and generalization in threat detection. The framework first establishes a hierarchical structure of attack scenarios and types, leveraging a graph attention network to encode semantic and structural dependencies among labels. Subsequently, the classification task is reformulated as a masked language modeling problem through prompt learning, enabling effective semantic alignment. Furthermore, a novel adversarial training mechanism is proposed, which utilizes local hierarchical information as a potent regularization signal. Within a game-theoretic architecture comprising a generator, an encoder, and a discriminator, the encoder is steered to integrate authentic hierarchical priors and produce high-fidelity oracle representations. This process encourages the generator to implicitly assimilate sample-specific hierarchical knowledge during adversarial learning, thereby improving the model's resilience to noisy inputs and its capacity to detect infrequent attacks. Evaluation results show that this method achieves an accuracy of 0.9972–1.0000 in complex mixed scenarios for scenario detection and 0.9987–0.9999 for attack category detection, while exhibiting strong robustness against interference.
The primary challenge in incremental learning for deep neural networks (DNNs) lies in balancing memory stability with learning plasticity as they incrementally acquire new knowledge. Existing incremental learning methods typically require storing portions of old datasets or relying on specialized network architectures to achieve learning without forgetting. Recent neuroscience findings reveal that hippocampal sharp-wave ripples play critical roles in consolidating neocortex memory by internally triggering the reactivation of spontaneous neural circuits without external stimulus. Inspired by this biological mechanism, we propose an incremental learning method termed Sharp-Wave Ripples Learning (SWRL), a novel incremental learning framework composed of two functionally complementary modules: a neocortex-like network for long-term memory and a hippocampus-like model for rapid new-knowledge encoding. SWRL continuously integrates newly acquired knowledge from the hippocampus-like module into the neocortex-like network via a meta-weighted memory integration mechanism guided by internally generated SWR-like signals. Our framework is task-agnostic, architecture-agnostic, and exemplar-free. Experimental results on multiple public benchmarks demonstrate that SWRL achieves superior incremental learning performance under various scenarios across diverse DNN architectures, requiring no access to previous data or modifications to the base architecture. The code and appendix are both released at https://github.com/SYVAE/SWRL.
Federated learning (FL) enables privacy-preserving video human activity recognition without centralizing data. Despite its potential, FL remains fundamentally constrained by the high communication overhead incurred during training. Existing methods, such as model quantization, gradient sparsification and low-rank approximation, aim to mitigate this overhead but often at the cost of model expressivity, leading to reduced classification accuracy. In this work, we take a representation-centric perspective and show that for video-base activity recognition using pretrained embeddings, class-discriminative information lies in a very low-dimensional subspace, that preserves linear seperability between activity classes. Motivated by this observation, we propose Federated discriminative subspace discovery, a framework that collaboratively identify and operate within such a subspace. Instead of communicating full classifier updates, each client first projects its local video embeddings using a shared random projection matrix. It then computes the covariance of the projected embeddings and transmits this statistic to the server for aggregation. The server reconstructs a global covariance matrix, performs singular value decomposition, and selects the top-k eigenvectors that define the global class-discriminative subspace. Subsequent communication occurs only in this compact subspace. Extensive experiments on UCF101, HMDB51, and Toyota-SmartHome demonstrate that, learning and sharing the classifier in the discovered subspace, reduces communication costs by up to 61%, with a moderate drop in activity recognition accuracy compared to the full model.
Federated Learning (FL) faces two major challenges when deploying large vision-language models such as contrastive language-image pretraining (CLIP) model, namely high computation/communication overhead and performance degradation due to data heterogeneity. To address these, we propose FedENC, a novel federated adaptive CLIP framework. FedENC keeps the CLIP encoder frozen and introduces a lightweight communicable global expert adaptation module for learning general knowledge across clients and two local experts for client-specific adaptation to reduce computation and communication overhead. To mitigate data heterogeneity, we introduce neural collapse (NC) alignment to encourage the model to learn unified and balanced image feature representations. To enrich the performance, we propose a cosine and Kullback–Leibler (KL) divergence based alignment function to improve the prediction consistency between the global and local experts. Finally, we ensemble their outputs for robust inference. Extensive experiments on six datasets (4 natural + 2 medical) in challenging settings (e.g., domain shift) show that FedENC improves the test accuracy (e.g., by +5.37% on DomainNet compared to LoRA) with reasonable communication load (e.g., 15.2x lower than LoRA, with 3.3GB GPU memory usage). The code is available at https://github.com/AIPMLab/FedENC.
Training with instance-dependent label noise poses a critical challenge in deep learning, severely impeding model generalization. Existing methods often struggle to break the vicious cycle where corrupted labels degrade feature representations, which in turn impairs noise identification. To overcome this, we propose the Meta-Cognitive Resonance Label Correction (MecReC) framework. For feature representation, MecReC structurally decouples representation learning from label correction, leveraging a pre-trained model as a frozen anchor to ensure feature robustness. For label correction, we introduce a novel meta-cognitive resonance mechanism that aggregates multiple complementary signals to comprehensively assess label reliability. These collaborative signals are fed to a lightweight meta-weight network, which automatically learns whether and how to correct each label. Furthermore, our personalized confidence thresholding strategy guarantees a tighter error bound for label purification. Extensive experiments demonstrate that MecReC consistently outperforms SOTA methods, exhibiting superior robustness particularly under extreme noise conditions. Code and additional details are available at https://github.com/JieSu272/MecReC.
Efficient transfer learning methods for large-scale vision–language models (e.g., CLIP) enable strong few-shot transfer, yet existing adaptation methods follow a fixed fine-tuning paradigm that implicitly assumes a uniform importance of the image and text branches, which has not been systematically studied in image classification. Through extensive analysis, we reveal a Branch Bias issue in vision–language image classification: adapting the image encoder does not always improve performance under out-of-distribution settings. Motivated by this observation, we propose A₃B₂, an Adaptive Asymmetric Adapter that alleviates Branch Bias in few-shot learning. A₃B₂ introduces Uncertainty-Aware Adapter Dampening (UAAD), which automatically suppresses image-branch adaptation when prediction uncertainty is high, enabling soft and data-driven control without manual intervention. Architecturally, A₃B₂ adopts a lightweight asymmetric design inspired by mixture-of-experts with Load Balancing Regularization. Extensive experiments demonstrate that A₃B₂ consistently outperforms 11 competitive prompt- and adapter-based baselines.
Gradient inversion attacks (GIAs) have shown that shared gradients in federated learning leak private training data. However, current textual GIAs fail in large batch size, as simply increasing batch size can serve as a stable defense against such attacks. In this paper, we propose CDO-GIA, a novel textual gradient inversion attack via continuous-discrete optimization that extends the maximum effective batch size from 4 to 8 for reconstructing text from gradient. In continuous optimization phase, CDO-GIA incorporates a dynamic weight decay mechanism to alleviate embedding homogenization caused by embedding regularization at large batch sizes, thereby enhancing unigrams reconstruction accuracy. Moreover, our method introduces a tabu beam search mechanism guided by the language model prior during discrete optimization. This design facilitates fine-grained exploration of high-dimensional token spaces via precise token order adjustment, thus reconstructing semantically coherent sequences. By alternating optimization between continuous and discrete phases, CDO-GIA effectively doubling practical attack limit of prior textual GIAs. Extensive experiments are conducted on three binary text classification datasets. The experimental results indicate that CDO-GIA surpasses all baseline methods. With a batch size of 8, it achieves performance gains of 25.01%, 155.64%, and 22.72% over the baselines in terms of Rouge-1, Rouge-2 and Rouge-L scores.