The rapid advancement in generative artificial intelligence have enabled the creation of 3D human faces (HFs) for applications including media production, virtual reality, security, healthcare, and game development, etc. However, assessing the quality and realism of these AI-generated 3D human faces remains a significant challenge due to the subjective nature of human perception and innate perceptual sensitivity to facial features. To this end, we conduct a comprehensive study on the quality assessment of AI-generated 3D human faces. We first introduce Gen3DHF, a large-scale benchmark comprising 2,000 videos of AI-Generated 3D Human Faces along with 4,000 Mean Opinion Scores (MOS) collected across two dimensions, i.e., quality and authenticity, 2,000 distortion-aware saliency maps and distortion descriptions. Based on Gen3DHF, we propose LMME3DHF, a Large Multimodal Model (LMM)-based metric for Evaluating 3DHF capable of quality and authenticity score prediction, distortion-aware visual question answering, and distortion-aware saliency prediction. Experimental results show that LMME3DHF achieves state-of-the-art performance, surpassing existing methods in both accurately predicting quality scores for AI-generated 3D human faces and effectively identifying distortion-aware salient regions and distortion types, while maintaining strong alignment with human perceptual judgments. Both the Gen3DHF database and the LMME3DHF will be released upon the publication.
论文检索
输入标题、作者或关键词，从 2,111 篇学术成果中精准定位
Transfer-based adversarial attacks have endowed adversarial examples with the ability to transfer from a source model to an unknown target model, which poses a more realistic threat to security-critical applications. Existing transferable adversarial attacks generally suffer from overfitting to the source model, i.e., the perturbations are locally optimal in the source model and focus on the model-specific information. We demand the adversarial perturbation to contain more generalized knowledge, which reveals the intrinsic general properties and can introduce model-general optimum into adversarial examples, for improving transferability. To this end, we devise a Bi-level Bias Mitigated Attack (BBMA), which empowers the transferability of adversarial examples by exploring generalization in two levels: 1) Progressive filtering of high-frequency sample components. We first propose to remove the sample-specific high-frequency components of samples to explore model-level generation. To simulate how a model evaluates feature importance at different stages, we devise a stride-wise step-tuning strategy to progressively produce multiple samples for aggregating the gradients. 2) Accumulated gradient-guided model attention shift. To facilitate the sample-level bias mitigation, we employ an accumulated gradient-guided attention map to distort the more generalized features during perturbation generation. Comprehensive experiments on several benchmarks demonstrate the superiority of our method in attack transferability over state-of-the-art attacks.
Facial image steganography is crucial for privacy-preserving media transmission. Traditional embedding methods degrade image quality and are vulnerable to steganalysis, while GAN-based non-embedding approaches lack controllability and realism. Diffusion-based methods using textual prompts face two key issues: (1) security risks from interpretable prompts and (2) poor preservation of facial details. This paper presents Featurized Denoising Diffusion Implicit Models (F-DDIM), a novel non-embedding steganography framework. First, F-DDIM replaces explicit textual prompts with implicit image-based encoding, enhancing security. Second, it selectively refines facial regions for natural and high-quality recovery through iterative reconstruction. Third, it enables indistinguishable encryption without secret key sharing via a novel sub-code embedding algorithm. Fourth, a refinement step post-decoding improves the clarity and accuracy of recovered facial image details. Experimental results demonstrate that F-DDIM achieves superior image fidelity and robustness against transmission interference.
Multimodal learning integrates diverse modalities to enhance robustness, yet real-world scenarios suffer from heterogeneous imbalance phenomena (noise interference, modality partial missing, intermodal information disparities), degrading performance through biased feature representations. Existing methods fail to adaptively modulate models under dynamic imbalance conditions. We propose GMML, a framework dynamically balancing multimodal gradients to counteract imbalance-induced biases: i) An imbalance-aware gradient modulation adaptively identifies contributions with smooth weight transitions to balance conflicting gradients; ii) A parameter constraint method enforces ℓ2-norm constraints on encoders, suppressing parameter oscillations and blocking noisy updates under modality missing/noise. Theoretically, GMML achieves a larger certified radius upper bound for complex imbalances, with convergence radius analysis providing theoretical guarantees. Experiments demonstrate superior robustness against three imbalance types, outperforming state-of-the-art by 3.3% and 2.3% in accuracy on KS and UCF-101 benchmarks. series Code: https://github.com/zhangzikai-security-ML/GMML.
Deep neural networks on cloud platforms face growing security threats, with AI services increasingly relying on heterogeneous models for the same task to meet diverse user needs. Existing methods fail to distinguish benign modifications from malicious attacks in cross-model scenarios. To address this challenge, we propose a non-intrusive cross-model watermarking method that generates discriminative samples as universal keys, enabling authentication without altering model parameters or architectures. Specifically, we introduce a margin enhancement loss to amplify confidence gaps between benign and malicious behaviors, ensuring high transferability across models. Both theoretical analysis and experimental results demonstrate the high efficacy of our proposed method. The generated samples maintain high visual fidelity (SSIM > 0.99), achieve over 3 times higher discriminability than existing methods, retain over 93% accuracy under benign modifications, and detect malicious attacks with accuracy dropping below 9%. Overall, our proposed method provides a robust, transferable, and non-intrusive solution for cross-model authentication, making it ideal for real-world applications where security is critical.
Micro-expressions (MEs) are involuntary facial expressions that reveal genuine emotions and have significant applications in fields such as psychology, security, and human-computer interaction. However, previous ME datasets are mainly collected in controlled laboratory environments, such as fixed views, single illumination and head movements, limited subjects and the lack of background. There are significant gaps between them and the real world. To handle this issue, we introduce a novel Natural Micro-Expression (NaME) dataset, a natural dataset collected under unconstrained real-world conditions. It encompasses (1) diverse subjects, multiple views and varying head movements ; (2) rich background information, providing a more realistic benchmark for the micro-expression recognition (MER) research. Furthermore, we propose a MER benchmark for natural environments, named MixFormer. MixFormer includes an efficient sparse attention mechanism to capture subtle facial motions from various factors, and a face-background mix of attention module to model the environment context to help MER. Extensive experiments are conducted to analyze our NaME dataset and benchmark. We believe that our dataset and benchmark will pave the way for future research in MER beyond controlled settings, facilitating the deployment of MER in practical applications. NaME is available at github.com/real-ljt/NAMEdataset.
Advances in automatic graphical user interface (GUI) agents have brought significant privacy and security challenges, especially cloud-based solutions that may leak sensitive data and be vulnerable to man-in-the-middle attacks. To address these issues, we propose MMPro, a novel GUI agent framework that adopts a separated perception-thinking-execution architecture. The perception module and the execution module process inputs locally to generate outputs to ensure security, and the thinking module operates on abstract representations to ensure the effectiveness of the GUI agent.By modularizing each stage while introducing a hybrid description language (HDL), MMPro transforms screen images into abstract structured representations, minimizing the risk of sensitive information leakage. Experimental results on the OSWorld benchmark show that MMPro outperforms existing GUI agent methods while ensuring strong privacy protection and real-time interaction capabilities. This work presents a pioneering approach to developing efficient, privacy-conscious, and explainable automated GUI agents.
Recent advancements in image generation have provoked social and security concerns, yet most detection methods rely on black-box models that generalize poorly. By utilizing advances in Multi-modal Large Language Models (MLLMs), we propose a framework that fuses six specialized paradigms, each analyzing a distinct aspect of the image, to provide a final verdict with coherent, evidence-based reasoning. Experimental results on a diverse dataset of real and AI-generated images demonstrate that our approach outperforms both traditional detection methods and top humans, while providing . This study underscores the potential of MLLMs in developing robust, explainable, and reasoning-driven detection systems. The code is available at https://github.com/Gennadiyev/mllm-defake.
Unsupervised domain adaptation transfers knowledge from a labeled source domain to an unlabeled target domain, which assumes that the source domain is labeled correctly, and both source and target data are available during the adaptive process. But collecting large-scale datasets with fully precise annotations is expensive and time-consuming. Besides, due to data privacy and security issues, the source data is often inaccessible during domain adaptation and only unlabeled target data is available. Therefore, considering both the source domain data with noisy labels and the unavailability of source domain data during domain adaptation, this paper proposes an Adaptive Neighbors and Uncertainty Estimation (ANUE) method for Source-Free Unsupervised Domain Adaptation with Noisy Labels (SF-UDA-NL). To the best of our knowledge, there has been no prior research conducted on this particular issue. Specifically, since source domain data contains noisy labels, we propose a reweighted small-loss with uncertainty estimation to filter reliable samples for updating the dual-branch network. To prevent noise knowledge from misleading domain adaptation, we adopt a contrastive learning framework and design an adaptive neighbors module to help target samples generate more reliable pseudo-labels. Then, we design a reweighted contrastive loss at both class level and instance level based on uncertainty estimation to further enhance the network's classification performance. We conduct extensive experiments on three widely used datasets for unsupervised domain adaptation in image classification, and the results demonstrate the effectiveness and robustness of our method.
Federated Prompt Tuning (FPT) integrates large pre-trained Vision Transformers (ViT) into Federated Learning (FL) by leveraging Visual Prompt Tuning (VPT), achieving state-of-the-art performance with enhanced efficiency across various visual downstream tasks. However, data heterogeneity, such as feature shift and class imbalance, limits prompts' transferability and robustness in FPT. Existing methods primarily focus on generalized FPT (GFPT) or personalized FPT (PFPT), while only a few methods make initial attempts to integrate both approaches. In this paper, we propose a new FPT framework, dubbed DualFPT, which handles data heterogeneity from both generalized and personalized perspectives. Specifically, DualFPT divides the learnable prompts into global and local prompts to jointly capture general and client-specific information, achieving the harmonization of GFPT and PFPT. The generalization of DualFPT is realized by the Feature Sharing (FS) mechanism, which effectively narrows the distribution gap by allowing clients to securely share a portion of sensitive features. The key feature for improving personalization is the Prompt Composition Scheme (PCS), which weights local prompts with distribution similarity to generate composite prompts, thus achieving automatic distribution adaptation. Extensive experiments under feature shift and class imbalance scenarios demonstrate the superior performance of DualFPT. On DomainNet and CIFAR-100 (D (0.1)), DualFPT surpasses SGPT by 4.35% and 5.89% for generalization along with 7.89% and 9.09% for personalization. Ablation studies further validate the effectiveness, efficiency, and security of DualFPT.
With the popularization of the Internet and the diversification of attack methods, web security has become an important part of information security. As a carrier of network behavior, traffic can reveal attack behaviors in the Web environment through malicious traffic detection. Since images can fully express spatial features and local associations, it is feasible to visualize traffic as images and retrieve key feature information to detect malicious traffic. However, existing methods are prone to redundancy during feature extraction. Secondly, a single perspective makes it difficult to learn patterns with universality from diverse feature information. In addition, the selection of segmentation thresholds in the preprocessing is closely related to the model's information retrieval effect. The commonly adopted preset thresholds are difficult to cope with the changes in traffic data, limiting the applicability of the existing methods. Therefore, this paper proposes a Multi-Module-Based Composite Robust Model for Network Attack Detection (MCNAD). The model adopts depthwise separable convolution (DSC) to reduce redundant information, proposes a multi-scale feature learning module to enhance the model characterization ability, and proposes a gray level co-occurrence matrix segmentation algorithm with adaptive threshold (GLCM-AT) to optimize data preprocessing. The results show that MCNAD improves detection performance with better detection efficiency, generalization ability, and robustness, demonstrating its wide applicability in multiple scenarios.
Cross-domain graph anomaly detection (GAD) aims to identify nodes that significantly deviate from normal patterns in unseen target domains, showing great potential in applications such as multimedia content security and financial risk control. However, existing methods often rely on semantic information trained on individual datasets, which makes it difficult to capture node commonalities across domains and limits generalization in complex multimedia environments. To address these challenges, we propose Zero-GAD, a universal Zero-shot Graph Anomaly Detection framework tailored for cross-domain scenarios. Zero-GAD leverages a novel de-semanticized strategy to train a unified detection model that can be directly applied to unseen domains without retraining or fine-tuning. The framework is built upon two key components: (1) a Global Information Unification Module, which projects graph data into the spectral domain and performs normalization to align the energy distribution in the frequency space; and (2) a Node-Neutralized Discrepancy Scoring Module that leverages the discrepancy between the original and reconstructed node representations to produce effective anomaly scores. Extensive experiments show that Zero-GAD achieves superior accuracy compared to existing models under a GAD setting.
Multi-modal visual tracking leverages complementary sensor information to enhance robustness under challenging conditions. However, the security of multi-modal tracking systems remains largely unexplored. Existing attacks primarily target single-modal trackers or independently disrupt each modality, failing to exploit the inherent feature interactions and fusion mechanisms that define multi-modal tracking. As a result, these methods exhibit limited attack effectiveness and fail to assess multi-modal tracking systems' vulnerabilities accurately. Understanding these security risks is crucial, as adversarial threats could lead to severe failures in safety-critical applications. To address these challenges, a feature-aware adversarial attack, termed FA3T is proposed. It is designed to explicitly disrupt feature extraction and cross-modal alignment, thereby weakening the fusion process that multi-modal trackers rely on. To achieve this, a Frequency-Spatial Feature Separation (FSFS) module is constructed to perturb feature representations at multiple levels, weakening the modality-complementary advantages of multi-modal tracking. Furthermore, a Target Confusion Attack (TCA) module is devised to manipulate the target-background-template relationships, making it increasingly difficult for the tracker to distinguish the true target, significantly impairing tracking performance. Extensive experiments on five benchmark datasets (i.e., LasHeR, RGBT234, DepthTrack, VOT-RGBD2022, VisEvent) across three different modalities (RGB-T, RGB-D, and RGB-E) demonstrate that our attack substantially degrades state-of-the-art multi-modal trackers, exposing their susceptibility to adversarial threats.
The spread of tampered text poses a critical challenge to information security. Previous methods for tampered text detection (TTD) primarily relied on visual artifacts as clues, while overlooking potential semantic inconsistencies introduced during text manipulation. To address this limitation, we propose TVSIP (Tampered text Visual-Semantic InterPreter), a novel framework leveraging Multimodal Large Language Models (MLLMs) to integrate both visual and semantic clues for comprehensive tampered text analysis and verification. TVSIP consists of a Locator and an Interpreter. The Locator combines the visual detection ability of existing expert models with the semantic comprehension capabilities of MLLMs to create precise tampering masks. Subsequently, the Interpreter provides comprehensive descriptions and explanations based on identified tampered regions. To train and evaluate TVSIP, we construct the TextDDLE benchmark using GPT-4o. Extensive experiments demonstrate that TVSIP outperforms expert models in pixel-level localization and advanced MLLMs in interpretability. Furthermore, it maintains robustness against image degradation and exhibits strong generalization ability on out-of-domain datasets. Our work highlights the crucial role of semantic inconsistencies in TTD and establishes a more reliable verification system for ensuring document authenticity in the digital age.
Malicious Deepfakes pose serious security risks by producing highly realistic forged faces. While numerous countermeasures have been developed to train binary Deepfake classifiers, their limited generalization capacity restricts practical deployment. To proactively defend against Deepfakes, we propose SVS-WM, a Self-Verifiable Semantic Watermarking strategy. The core idea behind SVS-WM is to embed pairs of correlated watermarks within facial semantics, leveraging the inherent fragility of these features, i.e., any semantic modification will disrupt the watermark correlation, thereby enabling robust Deepfake detection. SVS-WM employs a facial semantic disentanglement and reconstruction network, allowing semi-fragile watermarks to be embedded concurrently across multiple semantic levels, including identity and multi-levels of attributes. Specifically, pairs of pseudo-random noise watermarks are adaptively injected into facial attribute and identity features. During propagation stage, the protected image may encounter identity or facial attributes manipulations, we then detect Deepfakes by verifying the correlation result between the decoded attribute watermark and the extracted identity vector. This unique cross-verification mechanism enables authentication without requiring original reference watermark, thereby realizing blind Deepfake detection. Extensive experiments validate the effectiveness of our approach, achieving an average detection accuracy of 98.19% across diverse Deepfake manipulations.
Knowledge graphs (KGs) are widely used to store multi-source and heterogeneous structural knowledge, making federated knowledge graph completion (FedKGC) a crucial research topic. FedKGC aims to complete distributed KGs while maintaining privacy and security. Existing FedKGC methods primarily rely on uni-modal structural embedding aggregation for global knowledge sharing, which suffers from the demanding assumption that intersecting entities exist across different clients and are known by the omniscient server. Meanwhile, these uni-modal structure-only methods neglect the exploitation of client-side multi-modal information. In this paper, we propose a new framework MuCo2 to kill two birds with one stone and facilitate client-server co-design through multi-modal codebooks (MuCo). Moving beyond the traditional structure-only paradigm, we introduce multi-modal information of entities as the foundation for KGC modeling and communication. We design a MuCo-based fine-grained KGC model on the client and a MuCo-based communication mechanism on the server, which does not require entity mapping in global aggregation anymore. Comprehensive experiments demonstrate the effectiveness, generalization, reasonability, efficiency, and explainability of MuCo2.
Temporal Intention Localization (TIL) is crucial for video surveillance, focusing on identifying varying levels of suspicious intention to enhance security monitoring. However, existing discrete classification methods fail to capture the continuous progression of suspicious intentions, limiting early intervention and explainability. In this paper, we reconceptualize hidden intention modeling by shifting from discrete classification to continuous regression and propose Suspicion Progression Analysis Network (SPAN), which capture the fluctuations and progression of hidden intentions over time. Specifically, when analyzing the temporal progression of suspicion, we discover that suspicion exhibits long-term dependency and cumulative effects across extended sequences, characteristics significantly similar to the settings in Temporal Point Process (TPP) theory. Based on these insights, we formalize a suspicion score formula that models continuous changes while accounting for temporal characteristics. We also propose Suspicion Coefficient Modulation to adjust suspicion coefficients using multimodal information, reflecting different effects of suspicious actions. Notably, we introduce a Concept-Anchored Mapping method to quantify associations between suspicious actions and predefined intention concepts, enabling understanding of not just actions occurring but also their potential underlying intentions. Extensive experiments on the HAI dataset show that SPAN significantly outperforms existing methods, reducing MSE by 19.8% and improving average mAP by 1.78%,. Notably, SPAN achieves a 2.74% mAP gain in low-frequency cases, indicating superior capability in capturing subtle behavioral changes.Compared to discrete classification systems, out continuous suspicion modeling method enables earlier detection and more proactive interventions, substantially enhancing both system explainability and practical utility in security applications.
Noise Matters: Diffusion Model-based Urban Mobility Generation with Collaborative Noise Priors
PDF ↗With global urbanization, the focus on sustainable cities has largely grown, driving research into equity, resilience, and urban planning, which often relies on mobility data. The rise of web-based apps and mobile devices has provided valuable user data for mobility-related research. However, real-world mobility data is costly and raises privacy concerns. To protect privacy while retaining key features of real-world movement, the demand for synthetic data has steadily increased. Recent advances in diffusion models have shown great potential for mobility trajectory generation due to their ability to model randomness and uncertainty. However, existing approaches often directly apply identically distributed (i.i.d.) noise sampling from image generation techniques, which fail to account for the spatiotemporal correlations and social interactions that shape urban mobility patterns. In this paper, we propose CoDiffMob, a diffusion model for urban mobility generation with collaborative noise priors, we emphasize the critical role of noise in diffusion models for generating mobility data. By leveraging both individual movement characteristics and population-wide dynamics, we construct novel collaborative noise priors that provide richer and more informative guidance throughout the generation process. Extensive experiments demonstrate the superiority of our method, with generated data accurately capturing both individual preferences and collective patterns, achieving an improvement of over 32%. Furthermore, it can effectively replace web-derived mobility data to better support downstream applications, while safeguarding user privacy and fostering a more secure and ethical web. This highlights its tremendous potential for applications in sustainable city-related research. The code and data are available at https://github.com/tsinghua-fib-lab/CoDiffMob.
Nowadays, Graph Fraud Detection (GFD) in financial scenarios has become an urgent research topic to protect online payment security. However, as organized crime groups are becoming more professional in real-world scenarios, fraudsters are employing more sophisticated camouflage strategies. Specifically, fraudsters disguise themselves by mimicking the behavioral data collected by platforms, ensuring that their key characteristics are consistent with those of benign users to a high degree, which we call Adaptive Camouflage. Consequently, this narrows the differences in behavioral traits between them and benign users within the platform's database, thereby making current GFD models lose efficiency. To address this problem, we propose a relation diffusion-based graph augmentation model Grad. In detail, Grad leverages a supervised graph contrastive learning module to enhance the fraud-benign difference and employs a guided relation diffusion generator to generate auxiliary homophilic relations from scratch. Based on these, weak fraudulent signals would be enhanced during the aggregation process, thus being obvious enough to be captured. Extensive experiments have been conducted on two real-world datasets provided by WeChat Pay, one of the largest online payment platforms with billions of users, and three public datasets. The results show that our proposed model Grad outperforms SOTA methods in both various scenarios, achieving at most 11.10% and 43.95% increases in AUC and AP, respectively.
In China, the number of riders in the on-demand delivery industry has surpassed ten million. Ensuring that these riders earn a decent income can enhance their financial security, reduce poverty, and promote social equity and stability. Due to ease of use, lower-cost maintenance and environmental friendliness, electric bicycles (e-bikes) are the primary mode of transportation for delivery riders. However, these riders frequently encounter depleted batteries due to limited capacity and prolonged charging times, necessitating inconvenient swaps or recharges during deliveries. To address this issue, we propose the e-bike Battery Swap-as-a-Service (eBaaS), an innovative battery-swapping system that leverages an intelligent AIoT network for seamless battery swapping at distributed locations across urban areas. eBaaS integrates edge-cloud collaboration, battery resource allocation, battery anomaly detection, and battery range prediction to minimize downtime and reduce unnecessary mileage. While eBaaS's potential benefits are evident, there has been a lack of robust methods to quantify its impact. Thus, we further developed the eBaaS Impact Evaluation Method (EIEM), the first comprehensive model to address this gap. EIEM analyzes data from approximately 260,000 delivery riders and 5 million riding trajectories. Findings indicate that eBaaS reduces average invalid mileage by 6 km and increases the order volume by an average of over 20% daily per e-bike rider. Meanwhile, the annual electricity savings result in a reduction of 2.74 million kilograms of carbon emissions for 260,000 riders. The eBaaS system is therefore significantly beneficial for environmental conservation and sustainable urban development.