Data-free knowledge distillation enables model distillation without original training data, which is critical for privacy-sensitive tabular domains. However, existing methods do not perform well on tabular data because they do not explicitly address feature interactions, which are critical for encoding predictive knowledge. We identify interaction diversity, systematic coverage of feature combinations, as an important factor for effective tabular model distillation. To operationalize this insight, we propose TabKD, which learns adaptive feature bins aligned with teacher decision boundaries, then generates synthetic queries that ensure uniform pairwise interaction coverage. Across 4 benchmark datasets and 4 teacher architectures, TabKD achieves the highest student-teacher agreement in 14 of 16 configurations, outperforming 5 state-of-the-art baselines. We further show that interaction coverage strongly correlates with distillation quality, validating our core hypothesis. Our work establishes interaction-focused exploration as a principled framework for tabular model distillation.
论文检索
输入标题、作者或关键词，从 3,069 篇学术成果中精准定位
Skeleton-based human activity recognition (HAR) has achieved strong empirical performance, yet most existing models remain black boxes and difficult to interpret. In this work, we introduce a neurosymbolic formulation of skeleton-based HAR that reframes action recognition as concept-driven first-order logical reasoning over motion primitives. Our framework bridges representation learning and symbolic inference by grounding first-order logic predicates in learnable spatial and temporal motion concepts. Specifically, we employ a standard spatio-temporal skeleton encoder to extract latent motion representations, which are then mapped to interpretable concept predicates via a spatio-temporal concept decoder that explicitly separates pose-centric and dynamics-centric abstractions. These concept predicates are composed through differentiable first-order logic layers, enabling the model to learn human-readable logical rules that govern action semantics. To impose semantic structure on the learned concepts, we align skeleton representations with LLM-derived descriptions of atomic motion primitives, establishing a shared conceptual space for perception and reasoning. Extensive experiments on NTU RGB+D 60/120 and NW-UCLA demonstrate that our approach achieves competitive recognition performance while providing explicit, interpretable explanations grounded in logical structure. Our results highlight neurosymbolic reasoning as an effective paradigm for interpretable spatio-temporal action understanding.
Federated learning (FL) enables collaborative model training without centralizing raw data, but privacy regulations such as the right to be forgotten require FL systems to remove the influence of previously used training data upon request. Retraining a federated model from scratch is prohibitively expensive, motivating federated unlearning (FU). However, existing FU methods suffer from high unlearning overhead, utility degradation caused by entangled knowledge, and unintended relearning during post-unlearning recovery. In this paper, we propose FedCARE, a unified and low overhead FU framework that enables conflict-aware unlearning and relearning-resistant recovery. FedCARE leverages gradient ascent for efficient forgetting when target data are locally available and employs data free model inversion to construct class level proxies of shared knowledge. Based on these insights, FedCARE integrates a pseudo-sample generator, conflict-aware projected gradient ascent for utility preserving unlearning, and a recovery strategy that suppresses rollback toward the pre-unlearning model. FedCARE supports client, instance, and class level unlearning with modest overhead. Extensive experiments on multiple datasets and model architectures under both IID and non-IID settings show that FedCARE achieves effective forgetting, improved utility retention, and reduced relearning risk compared to state of the art FU baselines. The source code can be found at https://github.com/thechosenchu/FedCARE.
The learnware paradigm supports model reuse by pairing each submitted model with a specification, a lightweight representation used by the learnware dock system to identify, match, and reuse models without accessing raw data. While specifications are essential for learnware identification, they are also data-dependent public artifacts and it is not clear whether they reveal private information. Recently, the Parameter Vector (PAVE) specification has been proposed and shown to be effective for learnwares, yet its privacy properties remain largely unexplored. In this paper, we provide the first theoretical privacy analysis for PAVE. Specifically, we first formalize two specification-induced risks in the learnware paradigm: the disclosure risk of the released specification and the amplification risk that the specification may strengthen attacks against the released model. Second, we characterize when compact PAVE releases admit intrinsic differential privacy (DP): under natural structural conditions of learnware docks, the compact PAVE specification satisfies an (ε, δ)-DP guarantee without explicit additive noise through a Gaussian-sketch view of stable parameter variations, and for regimes outside these conditions, we further provide DP-Stabilized-PAVE as a certified differentially private variant. Third, we show that the resulting DP guarantees control both disclosure risk and amplification risk, and we analyze the induced privacy--utility trade-off to guide effective learnware identification while preserving privacy.
Large EEG Models (LEMs) are drawing increasing attention in EEG, as large-scale pretraining yields transferable representations that improve generalization. As EEG research moves to real-world deployment, objectives and paradigms diversify, yielding increasingly heterogeneous and unevenly scaled datasets across institutions. This evolution induces distribution drift and scale imbalance, making static pretrained LEMs brittle and requiring adaptation to newly collected data. Under the constraints of the high cost of retraining and the privacy sensitivity of newly collected data, federated learning offers a practical route to update LEMs via decentralized parameter updates. Yet standard federated fine-tuning is unstable for pretrained LEMs, suffering backbone drift, misaligned updates under non-IID heterogeneity, and scale imbalance. We propose FedGLoRA: we freeze the backbone and communicate only LoRA adapters via a dual-branch design, with a shared global branch and a private on-device branch. FedGLoRA aggregates global updates on a Grassmann subspace with a Grassmann-proximal constraint, and uses bidirectional distillation to mitigate imbalance. Experiments on motor imagery and emotion recognition show consistent gains and improved stability over strong baselines.
Large language models (LLMs) dominate both everyday and specialized applications, but their high computational demand, energy consumption, and privacy risks are increasingly critiqued. Small language models (SLMs) mitigate these drawbacks and are gaining momentum in scenarios where full LLM capabilities are not required, such as agents, industrial systems, or edge devices. Nevertheless, a systematic comparison of model capabilities, energy usage, and scaling behavior has not been conducted yet. We evaluate 70+ SLMs from 2023–2025 on five task-specific benchmarks and compare them with two popular LLMs, revealing key trade-offs between energy, performance, and model selection. Our findings challenge common assumptions: First, smaller models are not automatically more efficient, and energy increases do not guarantee performance gains. Second, newer SLMs show clear improvements in performance–energy trade-offs, though the progress begins to plateau. Last, the efficiency landscape forms a clear Pareto frontier: initial energy increases yield substantial gains, but the last percentage points of performance need orders of magnitude more energy. These results highlight diminishing returns of scaling and emphasize the need for informed, task-aware model selection rather than size-driven choices.
Privacy concerns in distributed learning often lead clients to return intentionally altered gradient information. We consider the problem of learning convex and L-smooth functions under adversarial gradient perturbation, where a client's gradient reply to a server query can deviate arbitrarily from the true gradient subject to a distance bound. Our study focuses on two fundamental questions: (i) what is the smallest achievable sub-optimality gap (i.e., excess error in optimization) under such responses, and (ii) how many queries are sufficient to guarantee a given sub-optimality gap? We establish tight feasibility thresholds on the sub-optimality gap and provide algorithms that achieve these thresholds with provable query complexity guarantees.
Machine unlearning (MUL) aims to remove the influence of specific data from a trained model for data privacy and model adaptability. Existing MUL methods mostly assume the internal parameters and the training data of the target model are accessible. Nevertheless, in most practical scenarios, the model provider (MP) and the service operator (SO) are different entities with unequal model access privileges. The MP provides the model, while the SO can only access the model via APIs when handling unlearning requests. Under such an asymmetric access constraint, we propose One-Turn Knockout (OTK), a novel traceable and editable MUL framework based on a model-agnostic and editable proxy. Specifically, OTK first compresses the representation space of the target model into a discrete proxy based on codebook, with merely one pass post-training. Each data sample is recorded in the proxy space as a distribution over the codebook tokens, and its contribution to the model prediction can be cumulatively estimated via additive token statistics. Based on the traceable and editable proxy, the SO can instantly handle unlearning requests by (i) estimating the token distribution of the forgotten data, (ii) identifying the causal tokens, and (iii) erasing their contributions without the access to the model parameters and training data. Extensive experiments on multiple datasets and tasks show that OTK consistently outperforms state-of-the-art unlearning methods.
Split Federated Learning (SFL) has emerged as a pivotal paradigm for privacy-preserving distributed training on resource-constrained edge devices by partitioning neural networks between clients and a server. A critical design choice in SFL is the split layer, which determines the computation distribution and the semantic level of smashed data, directly impacting communication overhead, training latency, and model accuracy. Existing SFL methods largely ignore the time-varying nature of edge resources, relying on static resource profiles that lead to suboptimal efficiency and compromise model convergence. In this paper, we propose Dynamic Adaptive Split Federated Learning (DASFL), a unified framework that jointly addresses dynamic resource heterogeneity and non-IID data distributions. We employ a resource-aware dynamic split layer selection strategy that enables each client to minimize per-round latency by adapting to instantaneous local conditions. We also design an adaptive masked aggregation that robustly synchronizes client updates under split-induced structural heterogeneity. Extensive experiments on multiple benchmark models and datasets, conducted under heterogeneous, time-varying resource conditions and non-IID data distributions, demonstrate that DASFL achieves a superior accuracy-efficiency trade-off and faster convergence compared to state-of-the-art SFL baselines.
Federated Reinforcement Learning from Human Feedback (RLHF) enables the collaborative alignment of Large Language Models (LLMs) while preserving privacy, yet it faces critical bottlenecks arising from data heterogeneity. Existing approaches typically rely on rigid client-level clustering, which overlooks intra-client heterogeneity and fails to adapt to the multifaceted needs of individual users. To address this, we propose FedPrism, a novel framework that shifts alignment granularity from the coarse client level to the precise instance level. Similar to an optical prism dispersing mixed light into distinct spectral components, FedPrism decomposes complex, intra-client heterogeneous data streams by dynamically routing individual samples to specialized experts based on semantic features. Crucially, we devise a Posterior-Guided Performance Distillation (PGPD) mechanism that leverages experts' actual training loss as self-supervision to autonomously refine routing policies without explicit labels. Extensive experiments show that FedPrism not only establishes new state-of-the-art results but also effectively mitigates the negative transfer prevalent in non-IID settings, ensuring superior alignment fidelity.
Graph Neural Networks (GNNs) have demonstrated strong predictive performance across a wide range of applications. However, their increasing deployment has raised critical fairness concerns, as these models can inherit and amplify existing biases. Most existing fairness approaches rely on explicit demographic information, either directly available or inferred, to measure and mitigate bias. In real-world settings, however, such information is often unavailable or legally prohibited to infer due to privacy concerns, legal restrictions, or regulatory constraints, which substantially limits the applicability of these methods. To address this challenge, we propose Demographic-Independent Fair Graph Learning (DIFGL), a novel framework for fair graph learning without demographic supervision. DIFGL mitigates group unfairness by minimizing disparities in individual treatment across implicitly identified subgroups, thereby enforcing fairness without requiring explicit demographic information. Extensive experiments on benchmark datasets demonstrate that DIFGL achieves significant improvements in fairness while maintaining competitive predictive performance.
Driven by privacy regulations, federated unlearning (FU) aims to remove the influence of specific clients or samples from a trained federated model, approximating the behavior of retraining from scratch without the target data. However, existing FU methods are largely reactive: retraining-based solutions are accurate but prohibitively expensive, while parameter-based approaches are more efficient yet may cause irreversible knowledge damage and catastrophic forgetting. We introduce PFU-SKLA, a proactive FU framework that endows models with built-in forgettability. During pretraining, we perform Orthogonality-guided Representation Disentanglement (ORD) to learn a robust, disentangled feature space that reduces cross-client/class interference. Additionally, we use Dynamic Sensitivity-based Key Layer Identification (DS-KLI) to identify sensitive layers that hold target knowledge. During unlearning, we propose a sparse, lightweight adaptation strategy that precisely erases target knowledge by inserting sparse adapters into the identified critical layers while preserving non-target knowledge by freezing the backbone. Extensive experiments across multiple datasets and three standard unlearning settings show that PFU-SKLA consistently approaches retraining-from-scratch performance, while substantially reducing communication and computation costs compared to state-of-the-art (SOTA) FU methods.
Large language models (LLMs) are increasingly used as general planners in embodied intelligence, enabling high level coordination and low level task planning for both single robot and multi-robot collaboration. This increasing reliance on embodied LLM planners also raises critical security concerns, since misaligned or manipulated instructions can be translated into physical actions. Prior work has studied such threats in single robot settings, while security risks in LLM controlled multi-robot collaboration, especially those propagated through inter robot communication, remain largely unexplored. To bridge this gap, we propose a novel attack paradigm for multi-robot system in which the adversary interacts with only a single entry robot. The compromised robot then propagates malicious intent through peer communication, leading to coordinated unsafe actions across the system. Our evaluation, covering high risk dimensions of dereliction of duty, privacy compromise, and public safety hazards, reveals a persistent safety alignment gap in multi-robot planners. We quantify this process with three metrics, obedience, infectiousness, and stealthiness. Experiments demonstrate both persistent attacker control and rapid propagation: obedience reaches 1.00 in the strongest cases, and infectiousness rises to 0.90. Notably, the attack is highly efficient, requiring as few as 3.0 rounds to compromise all the robots while maintaining a stealthiness score of 0.81. Such risks are amplified when robots must resolve trade offs in critical situations, such as emergencies or conflicts of rights, because the coordination mechanism can unintentionally allow adversarial instructions to override safety requirements. The code is available at https://github.com/TheFatInsect/InfectBot.
Graph-level anomaly detection (GLAD) is crucial for ensuring the reliability of graph-driven applications by identifying abnormal graphs that deviate from the majority. Considering the privacy concerns in distributed scenarios, federated graph-level anomaly detection (FedGLAD) has emerged as a promising solution to enable collaborative detection without sharing raw data. However, existing methods suffer from poor generalization due to the reliance on unrealistic synthetic anomalies and insufficient personalization capabilities under data heterogeneity. To address these challenges, we propose a novel Federated graph-level anomaly detection approach with Cluster-adaptIve GAted Reconstruction (FedCIGAR). Specifically, we design a reconstruction-based paradigm trained on normal graphs to avoid synthetic data. Furthermore, we introduce a client-side node contribution gating mechanism and a server-side sliding window-based clustering strategy to tackle data heterogeneity. Extensive experiments demonstrate that FedCIGAR achieves superior performance and robustness compared to state-of-the-art methods.
Federated learning (FL) has emerged as a promising paradigm for adapting large language models (LLMs) to distributed data. To mitigate the high communication and memory overhead, parameter efficient techniques such as Low Rank Adaptation (LoRA) are widely adopted. However, practical deployments often exhibit rank and data heterogeneity, making direct aggregation of LoRA updates biased and unstable. Existing approaches enforce a unified rank or align heterogeneous updates into a single shared subspace, which undermines personalization and accuracy. Moreover, under differential privacy (DP), adding noise to such mixed updates could perturb client specific directions that should remain local, resulting in utility loss. To address these issues, we propose Selective Decoupled Federated LoRA (SDFLoRA), a structure aware LoRA framework that decouples each client update into a shared component for updating and a private component that preserves client specific semantics. Subspace alignment and aggregation are applied selectively to the shared module, while private modules remain local. We further design low rank re-compression to maintain a fixed rank budget for the shared module. This structure supports privacy aware optimization by injecting DP noise exclusively into the shared module. Experiments on multiple benchmarks demonstrate that SDFLoRA beats federated LoRA baselines and exhibits strong robustness under privacy constraints.
Backdoor attacks pose a serious threat to deep reinforcement learning (DRL). Current defenses typically rely on reward anomalies to reverse-engineer triggers and model finetuning to remove backdoors. However, complex trigger patterns undermine their robustness, and fine-tuning entails high costs, limiting practical utility. To this end, we shift defense concerns to trigger-agnostic backdoor output behaviors and propose BehaviorGuard, an online behavior-based backdoor detection and mitigation framework for DRL. Specifically, we find that regardless of attacks, backdoored policies induce consistent shifts in action distributions to ensure reliable activation, leaving detectable traces in high-quantile regions and distribution tails, even in the absence of triggers. Based on this, we design a novel metric that captures behavioral drift in action distributions to identify and suppress backdoor actions at runtime. To our knowledge, this is the first online backdoor defense that counters attacks both in single- and multi-agent DRL. Evaluated across diverse benchmarks with different backdoor attacks, BehaviorGuard consistently surpasses prior methods in both efficacy and efficiency.
Unauthorized personalization based on diffusion models pose a severe and growing threat to digital privacy by enabling the unauthorized replication and exploitation of individual identities. Existing disrupting-based defenses primarily add invisible perturbations arbitrarily across the entire image space to disrupt the generation process. However, we reveal that these methods fundamentally overlook the spatio-temporal dynamics of the personalization process, resulting in inefficient optimization that fails to sufficiently disrupt the core identity encoding mechanism. To mitigate these limitations, we propose IdentityMask, a robust protection framework that shifts the paradigm from arbitrary confusion to precise, targeted feature corruption. By anchoring the perturbation on subject-specific semantics and prioritizing the most critical diffusion timesteps, our framework ensures the disruption is maximized precisely where the identity is encoded. Additionally, a novel manifold projection strategy is introduced to embed the adversarial signals into the intrinsic structure of the image, rendering the protection resilient against state-of-the-art purification. Extensive experiments across diverse datasets, personalization techniques, and defense settings demonstrate that IdentityMask consistently outperforms prior state-of-the-art approaches in both protection efficacy and robustness.
Systematic safety evaluation of large language models must uncover diverse policy violations under tight query budgets. However, most red-teaming methods optimize attack success rate and repeatedly probe a narrow set of vulnerabilities, yielding redundant failures and leaving rarer yet critical violation categories unexplored. Under fixed budgets, such inefficient exploration delays the first discovery and limits category coverage. To address these limitations, we propose the Budget-Efficient Adaptive Cognitive Offense Navigator (BEACON), a budget-aware safety testing framework that uses Cognitive-Guided Monte Carlo Tree Search to navigate the violation search space under fixed budgets. BEACON innovatively approaches safety testing as a budget-constrained failure discovery process, aiming to identify diverse safety violations as early as possible within a fixed query budget. It also provides an efficiency-oriented evaluation perspective that measures early discovery and harm category coverage under budget constraints. Experiments on standard benchmarks and frontier LLMs show that BEACON discovers failures earlier and achieves higher coverage across policy violation categories. These results underscore the value of evaluating safety testing through discovery efficiency rather than attack success rate alone. Warning: This paper contains examples of harmful language and images, and reader discretion is recommended.
Open cross-domain federated graph learning facilitates collaborative learning among clients from distinct graph domains while preserving privacy. However, severe structure and feature heterogeneity in open scenarios exacerbates the multiplicative amplification of structural and feature noises within graph neural network encoders, leading to significant negative transfer. Existing methods addressing partial heterogeneity are insufficient. We theoretically reveal this amplification mechanism and propose a novel learning method FedFINFO, which leverages full-informativeness shared knowledge. Specifically, we construct an aligned and general feature space integrating node semantic and structural roles. We proposed a counterfactual multi-view framework to explicitly learn a structure masker for extracting consensus subgraphs. These jointly suppress the mutual amplification of noises at the source. We also propose a history-aware adaptive multi-channel aggregation mechanism to enable dynamic and personalized sharing driven by channel similarity while preventing collaborative oscillation. FedFINFO effectively reconciles common knowledge sharing with personalized knowledge retention, significantly mitigating negative transfer. Extensive experiments under cross-dataset and cross-domain settings demonstrate the superiority of our method.
Network traffic analysis is crucial for maintaining the security of networks. Yet deploying accurate models on edge nodes remains challenging due to protocol diversity, complex traffic behaviors, and stringent resource constraints. Although recent deep learning models achieve strong performance, their dense architectures incur high inference latency, making them unsuitable for edge deployments. Mixture-of-Experts offers a promising solution through conditional computation, yet existing methods overlook latent inter-task relations and suffer from severe expert load imbalance, leading to expert collapse and suboptimal efficiency. To address these limitations, we propose TAMoE, a novel Task-Aware Mixture-of-Experts framework for multi-task network traffic analysis. TAMoE integrates task semantics into both routing and representation learning through a novel Task-Aware Routing, enabling the router to dynamically select experts conditioned on both traffic features and task information. Besides, to train stably usable and scalable multi-task models, we further design a collaborative multi-task training strategy that encompasses both a dataset mixed construction and a joint training process. Experiments on six public benchmarks show that TAMoE achieves state-of-the-art results. Deployment on an NVIDIA® Jetson AGX Orin edge node demonstrates that TAMoE reduces inference latency by 60.6% compared to dense models and lowers the Gini Coefficient from 0.391 to 0.275 relative to Multi-Router MoE, achieving more balanced expert utilization.