The rapid progress of identity-feature-based face-swapping technology has raised concerns about impersonation and privacy violations. Although proactive defenses aim to block identity extraction at the source, existing methods suffer from perceptible visual artifacts, poor generalization across diverse deepfake models, and vulnerability to post-processing techniques (e.g., diffusion purification, image compression, and transformations). This work proposes a robust, generalizable proactive face-swapping defense via semantic gradient divergence (SGD-Guard) to address these challenges. It introduces an integrated feature gallery that uses CLIP features and a generalized identity feature, obtained by iteratively refining heterogeneous identity features into a homogeneous representation. This framework facilitates our semantic distortion attack by leveraging consensus weighting to target specific facial attributes within a CLIP-identity joint embedding space, disrupting deepfake generation while preserving visual fidelity. Furthermore, to ensure robustness against purification and post-processing, this method incorporates a module that prioritizes critical transformations by exploiting directional discrepancies. Comprehensive experiments demonstrate that the method effectively defends against diverse face-swapping models with high cross-model transferability.
论文检索
输入标题、作者或关键词，从 100,903 篇学术成果中精准定位
Monocular 3D object detection is a cost-efficient alternative to multisensor systems, yet it remains fragile to multi-component adversarial attacks that perturb the image and tamper with camera calibration. Compounded distortions degrade 3D reasoning by disrupting the correspondence between the 3D geometry and 2D image plane. To address this problem, this work proposes MonoPure, a monocular 3D object detection framework that performs multi-component purification via disentangled and projective representations. MonoPure incorporates a disentangled purification and segmentation module that purifies the image data, with a target-region probability map steering diffusion-based purification to focus on task-relevant regions. In addition, MonoPure presents a 3D detection decoder that integrates 2D skeleton keypoints as object-level spatial cues, enabling occlusion-robust 3D detection. Finally, a projective calib-purification module restores compromised intrinsics by iteratively minimizing the reprojection error between projected 3D boxes and calibration-invariant 2D detection boxes. The experiments confirm that MonoPure outperforms prior detectors under multi-component attacks and occlusion.
Video captioning models convert frames into visual tokens and generate descriptions with large language models (LLMs). Since encoding all frames is prohibitively expensive, uniform sampling is the default choice, but it enforces equal temporal coverage while ignoring the uneven events distribution. This motivates a Learnable Frame Selector (LFS) that selects temporally diverse and event-relevant frames. LFS explicitly models temporal importance to balance temporal diversity and event relevance, and employs a stratified strategy to ensure temporal coverage while avoiding clustering. Crucially, LFS leverages caption feedback from frozen video-LLMs to learn frame selection that directly optimizes downstream caption quality. Additionally, we identify the gap between existing benchmark and human's cognition. Thus, we introduce ICH-CC built from carefully designed questions by annotators that reflect human-consistent understanding of video. Experiments indicate that LFS consistently improves detailed video captioning across two representative community benchmarks and ICH-CC, achieving up to 2.0% gains on VDC and over 4% gains on ICH-CC. Moreover, we observe that enhanced captions with LFS leads to improved performance on video question answering. Overall, LFS provides an effective and easy-to-integrate solution for detailed video captioning.
Bi-temporal Semantic Change Detection (SCD) is a fundamental analytical framework for capturing state transitions across various domains, with remote sensing being a key application area in this work. However, the prevailing paradigm of comparing two discrete temporal snapshots suffers from inherent limitations due to temporal discontinuity. These limitations lead to pronounced radiometric inconsistencies and pseudo-change noise in Bi-temporal SCD. To address these, we propose the Pseudo-Temporal Feature Fusion Network (PTF-Net), which reconceptualizes bi-temporal SCD as continuous pseudo-temporal evolution modeling. This novel paradigm facilitates the recovery of the missing temporal context between observation points. At its core lies an innovative Non-Linear Style Interpolation Mechanism, which projects discrete bi-temporal observations onto a smooth latent manifold to simulate plausible surface evolution dynamics. The mechanism synthesizes a sequence of semantically coherent intermediate representations, which inherently bridge the temporal gap and disentangles style variations from semantic changes. To fully exploit this synthesized sequence, we design a dedicated Temporal Dynamics Perception Branch. This branch employs efficient temporal interactions to robustly discriminate structural mutations from transient noise, effectively acting as a semantic filter. Extensive experiments on three public datasets reveal that PTF-Net consistently outperforms state-of-the-art methods, achieving comprehensive superiority across mIoU, Fscd, and SeK metrics.
Spatio-temporal forecasting is inherently bottlenecked by a phenomenon we term Spatio-Temporal Duality: the paradoxical coexistence of time-varying physical lags and instantaneous semantic synchrony. Constrained by passive coupling, existing architectures often employ indiscriminate spatial aggregation that fails to dynamically arbitrate interaction intensity based on traffic states, forcing models to rely on redundant deep stacking to approximate complex dynamics. We address this issue by introducing Time Arbitrated Spatial GNN (TAS-GNN), an efficient framework that arbitrates spatial interactions through time. Our model leverages deep temporal semantics to dynamically and proactively manage the aggregation of spatial domains, effectively decoupling physical connectivity from semantic relevance. In addition, to ensure model simplicity, we also introduce spectral decoupling via discrete wavelet Transform (DWT) to capture multiscale dependencies with minimal overhead. Experiments on three real-world datasets (PEMS04, 07, 08) show that TAS-GNN not only outperforms the current baseline model in accuracy, but also significantly improves the inference speed while reducing the number of parameters.
In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, where workloads are highly dynamic and sensitive to runtime thread configurations. Existing multithreaded strategies struggle to match thread resources before or during execution, causing resource contention, scheduling overhead, and reduced throughput. Through empirical analysis, we identify the ratio of task execution time to scheduling time as the key factor determining the optimal thread count. Building on this insight, we propose AutoThread, a hybrid adaptive thread-tuning method for mitigating simulation bottlenecks in RL inference. AutoThread employs a Physics-Informed Neural Operator (PINO) as a thread-count predictor and incorporates a finite-source M/M/1 queueing model to constrain and guide prediction, enabling fast and accurate estimation under dynamic workloads. It further performs load-aware online fine-tuning to compensate for prediction errors and refine resource allocation. Experiments show that AutoThread improves average speedup by 18.4% over static strategies, achieves average throughput of 1.7× and 1.8× that of XGBoost and Reinforcer, respectively, and reduces execution time by up to 83.8% compared with state-of-the-art methods. Our code and dataset are publicly available at https://github.com/suchenjm/AutoThread.
The impressive performance of large language models (LLMs) arises from their massive scale and heterogeneous module composition. However, this structural heterogeneity poses significant optimization challenges. While adaptive optimizers such as Adam(W) provide per-parameter adaptivity, they do not explicitly account for module-level gradient heterogeneity, often resulting in slower convergence, suboptimal performance, or occasional training instability. Existing approaches typically rely on manually tuned module-specific learning rates or specific optimization strategies, which are computationally costly and difficult to generalize across tasks or models. To establish a more principled approach, we first analyze the noise-damping behavior of Adam in high-noise modules and introduce \textbf{Module-wise Learning Rate Scaling via SNR (MoLS)}. MoLS estimates module-level SNRs to scale Adam updates, allowing automated module-wise learning rate allocation without manual tuning. Empirical results across multiple LLM training benchmarks demonstrate that MoLS improves convergence speed and generalization, achieving performance comparable to carefully tuned module-specific learning rates, while remaining compatible with memory-efficient training algorithms.
Stochastic gradient methods are central to large-scale learning, yet their generalization theory typically relies on independent sampling assumptions. In many practical applications, data are generated by Markov chains and learning is performed in a decentralized manner, which introduces significant analytical challenges. In this work, we investigate the stability and generalization of decentralized stochastic gradient descent (SGD) and stochastic gradient descent ascent (SGDA) under Markov chain sampling. Leveraging a stability-based framework, we characterize how Markovian dependence and decentralized communication jointly influence generalization behavior. Our analysis captures the effects of network topology, Markov chain mixing properties, and primal–dual dynamics. We establish non-asymptotic generalization bounds for both algorithms, extending existing results on Markov stochastic gradient methods to decentralized and minimax settings.
Federated Learning (FL) faces two major challenges when deploying large vision-language models such as contrastive language-image pretraining (CLIP) model, namely high computation/communication overhead and performance degradation due to data heterogeneity. To address these, we propose FedENC, a novel federated adaptive CLIP framework. FedENC keeps the CLIP encoder frozen and introduces a lightweight communicable global expert adaptation module for learning general knowledge across clients and two local experts for client-specific adaptation to reduce computation and communication overhead. To mitigate data heterogeneity, we introduce neural collapse (NC) alignment to encourage the model to learn unified and balanced image feature representations. To enrich the performance, we propose a cosine and Kullback–Leibler (KL) divergence based alignment function to improve the prediction consistency between the global and local experts. Finally, we ensemble their outputs for robust inference. Extensive experiments on six datasets (4 natural + 2 medical) in challenging settings (e.g., domain shift) show that FedENC improves the test accuracy (e.g., by +5.37% on DomainNet compared to LoRA) with reasonable communication load (e.g., 15.2x lower than LoRA, with 3.3GB GPU memory usage). The code is available at https://github.com/AIPMLab/FedENC.
Similarity-graph-based multi-view clustering is effective in capturing non-Gaussian cluster structures, yet it suffers from an inherent conflict between scalability and geometry-consistent cluster assignment. In particular, cluster labels are typically inferred via secondary clustering rather than being learned directly from the underlying data geometry. Although anchor-based extensions alleviate the computational burden, they introduce the challenge of anchor-number selection. To address these limitations, we propose Multi-view Clustering via Manifold Decomposition (MvMD), which directly infers cluster labels from multi-view data without explicit similarity graph construction or anchor selection. MvMD reformulates multi-view clustering as a unified multi-view regression problem, where cluster labels are optimized as model variables. To improve robustness against missing small clusters, a global balance regularization is incorporated. Meanwhile, local geometric consistency and the low-rank structure of cluster assignments are jointly enforced through a symmetric matrix factorization scheme, which is efficiently realized via a truncated SVD-based low-rank approximation. Extensive experiments on benchmark datasets validate the effectiveness and efficiency of the proposed MvMD. The code is available at https://github.com/Vince-Doit/MvMD.
In intraday stock return forecasting, existing Transformer methods face high computational complexity and do not explicitly handle noisy predictors. We propose TransAlpha, a light-weight Transformer variant tailored for cross section data to advance end-to-end forecasting methods with three novel components (Cross-Sectional Denoising for signal purification, Temporal Attention Gating for trend weighting, and Smart Pooling to avoid information loss) along with a multi-component hybrid loss function. We validate on full A-share market stock data and four key segments (CSI 300/500/1000/2000) with 400 proprietary alpha factors of 15-minute frequencies. TransAlpha outperforms state-of-the-art baselines in predictive power and portfolio profitability, indicating tailored Transformer models have significant practical value in quantitative trade.
Despite recent advances in training-free text-guided image editing using pretrained rectified flow models, achieving a balance between text adherence and background preservation remains a challenge. Existing methods typically treat image editing as an open-loop trajectory generation task, adopting either an inversion-based or inversion-free paradigm. However, neither can perfectly achieve this goal. Specifically, inversion-based approaches accumulate inversion errors without feedback, whereas inversion-free methods exhibit unstable trajectories due to the lack of constraints. To address this, we propose EKFEdit, a novel framework that formulates image editing as a sequential state estimation problem in a nonlinear dynamic system, which models inversion-free editing trajectories as the system’s prior dynamics and leverages an Extended Kalman Filter to adaptively fuse observations from external editing algorithms. EKFEdit enables adaptive trajectory correction, effectively combining the complementary strengths of the different paradigms. We instantiate this framework into two concrete approaches (EKFEdit-RF and EKFEdit-DNA) via different observation methods, respectively. Extensive experiments demonstrate that our approaches achieve new state-of-the-art performance compared to previous methods. Code is available at https://github.com/baiyeweiguang/EKFEdit.
Symbolic regression (SR) aims to recover compact and interpretable mathematical expressions from data. Genetic programming (GP) directly searches over symbolic structures, but its population dynamics can make it difficult to reliably preserve and accumulate useful subexpressions. In contrast, reinforcement learning (RL)-based SR has shown strong empirical performance, suggesting that learned sampling dynamics may capture useful regularities in symbolic search. Motivated by this contrast, we analyze expressions sampled during RL training and identify a recurring pattern, termed progressive subexpression reuse, where useful simple subexpressions emerge early, become increasingly frequent, and support the formation of more complex structures. Based on this observation, we propose Reinforcement Genetic Programming (RGP), a purely GP-based and non-RL framework that explicitly realizes a stage-wise retention--reintroduction loop through a dynamically maintained subexpression pool and pool-guided population initialization. Experiments on standard SR benchmarks show that RGP matches or outperforms strong baselines without RL policy training, suggesting that progressive subexpression reuse is an effective mechanism for SR search. We release our code at https://github.com/wuxiangdong586-max/RGP.
Training with instance-dependent label noise poses a critical challenge in deep learning, severely impeding model generalization. Existing methods often struggle to break the vicious cycle where corrupted labels degrade feature representations, which in turn impairs noise identification. To overcome this, we propose the Meta-Cognitive Resonance Label Correction (MecReC) framework. For feature representation, MecReC structurally decouples representation learning from label correction, leveraging a pre-trained model as a frozen anchor to ensure feature robustness. For label correction, we introduce a novel meta-cognitive resonance mechanism that aggregates multiple complementary signals to comprehensively assess label reliability. These collaborative signals are fed to a lightweight meta-weight network, which automatically learns whether and how to correct each label. Furthermore, our personalized confidence thresholding strategy guarantees a tighter error bound for label purification. Extensive experiments demonstrate that MecReC consistently outperforms SOTA methods, exhibiting superior robustness particularly under extreme noise conditions. Code and additional details are available at https://github.com/JieSu272/MecReC.
Knowledge graphs (KGs) serve as crucial symbolic knowledge sources for many downstream applications but often suffer from incompleteness. Recent efforts have attempted to integrate pre-trained language models and KGs for the knowledge graph completion (KGC) task. However, existing methods introduce substantial model complexity or rely on prompt-based solutions that inject priors at a shallow level, which do not exploit the factual knowledge inherently encoded in model parameters. To address these issues, we propose a novel framework that treats LLMs as parametric knowledge sources for KGC. Our framework performs LLM knowledge elicitation to extract factual knowledge from the model’s internal representations. Then, a cross-granularity representation alignment method transforms sentence-level representations into entity-level representations and aligns them within a unified space. Finally, a dynamic learning schedule balances alignment and expressiveness throughout training. Extensive experiments on multiple benchmark datasets show that our proposed method can be integrated with diverse KGC baselines and consistently improves link prediction in both standard and lifelong settings.
Federated multi-view clustering (FedMVC) has been widely used to discover latent structures in distributed multi-view data, yet most methods still rely on the restrictive assumption of independent and identically distributed (IID) data. In practice, non-IID distributions with partial and imbalanced categories cause clients to learn biased and local representations, leading to model bias and unstable federated training performance. To address these issues, we propose a FedMVC framework called Bridging Inter-View and Client Heterogeneity: Federated Multi-View Clustering under Non-IID Data (H²-FedMVC), which tackles both inter-view and client heterogeneity in mixed-view settings. We propose a leader-guided learning mechanism to address inter-view heterogeneity by enhancing complementary and discriminative feature learning, and an expert adaptive aggregation strategy to handle client heterogeneity by prioritizing well-matched experts in global updates. Theoretical analysis and experiments demonstrate superior performance in heterogeneous mixed-view FedMVC settings, with code provided at https://github.com/JiazhenWang-0522/H2-FedMVC.
Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with egocentric video to enable context-aware modeling of the driver's attention over time. Furthermore, to tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset that augments four existing driving datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for multimodal cognitive distraction detection. Our codes and CogDrive dataset resources are available at https://github.com/langzhang2000/EyeCue.
Structured output generation is increasingly critical for real-world AI systems, particularly in on-device settings where small language models (0.5B–8B parameters) must produce machine-executable outputs under strict latency and privacy constraints. Although constrained decoding provides formal guarantees of structural validity without retraining, its effectiveness across different tasks, models, and constraint formalisms remains insufficiently understood. We introduce StructureBench, a comprehensive benchmark for structured generation on edge devices, covering JSON and tool-call generation, code synthesis, mathematics, science, and domain-specific languages, and evaluating over 11 on-device language and vision–language models spanning 0.5B–8B parameters. StructureBench systematically compares prompt-based generation with three widely used constrained decoding frameworks—Outlines, XGrammar, and Guidance—and adopts decoupled metrics to separately assess structural validity and semantic correctness. Our experiments show that constrained decoding consistently enforces syntactic validity, but does not reliably improve semantic accuracy and may even degrade performance for smaller models or complex grammars. These findings reveal clear task- and model-dependent boundaries for effective constrained decoding and highlight the need for scenario-aware adaptation in on-device structured generation. We release StructureBench at https://github.com/Str-Ben/StructureBench.
Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute–object compositions by leveraging shared knowledge from seen primitives. While recent works have begun to incorporate test-time adaptation to mitigate performance degradation caused by label-space distribution shifts, they suffer from primitive-level knowledge underutilization and cross-modal semantic asymmetry, hindering robust understanding of novel compositions. To address these issues, we introduce MLDA, a test-time adaptation framework for CZSL that jointly derives multi-level prototypes and performs dynamic vision–language alignment. Specifically, we construct and progressively update composition- and primitive-level prototype sets to accumulate structured semantic knowledge. By augmenting test inputs with diverse visual perspectives and enriched compositional descriptions, MLDA leverages optimal transport to achieve consistent matching of semantically salient information across modalities. Furthermore, composition activation scores are dynamically updated over the test stream to emphasize informative compositions and suppress irrelevant ones. Extensive experiments are conducted to verify the effectiveness of the proposed method. The code is available at https://github.com/keepgoingjkg/MLDA.
Open‑vocabulary video instance segmentation (OV‑VIS) couples spatial‑temporal reasoning with language grounding, yet its adversarial robustness has remained unexplored. We present the Dual-Objective Triggers (DOT), the first transferable attack on OV-VIS that simultaneously exploits the vision–language coupling and temporal coherence. DOT deploys a Dual Semantic Perturbation Module that overlays two complementary triggers: a Semantic Suppression Trigger erases the alignment between the true object and the query, while a Plausible Replacement Trigger steers the tracker toward a phantom trajectory that is visually plausible and text‑consistent. To amplify cross‑model transferability without sacrificing perceptual fidelity, we introduce Phase‑Guided Adversarial Training, which injects perturbations primarily in the phase spectrum while blending amplitudes with clean references. Extensive experiments on four state‑of‑the‑art OV‑VIS implementations demonstrate that DOT reduces mAP by up to 69.3% and raises attack success rate by up to 98%, outperforming the strongest baselines by a factor of 1.6× on average, while maintaining a PSNR of 52.49 dB, thus exposing critical security vulnerabilities and laying a foundation for future research on robust and trustworthy vision–language systems.