Large Language Models (LLMs) have achieved remarkable progress in understanding and generation tasks, yet they remain highly susceptible to adversarial prompt attacks that bypass safety safeguards and induce the generation of harmful content. Existing pre-generation and post-generation defense methods typically rely on intent recognition, surface-level features, or external classifiers, rendering them vulnerable to evasion via subtle prompt perturbations while incurring substantial computational overhead. In this paper, we propose PADD (Prefix-based Attention Divergence Detector), a lightweight pre-generation defense mechanism that leverages internal model signals for robust attack detection. At its core, PADD prepends a lightweight safety prefix to the input prompt and compares attention distributions between the original and prefixed prompts. By transforming cross-prompt comparisons into self-comparisons via composite signals of attention divergence and attention plasticity, PADD achieves strong separability between adversarial and benign prompts with low-latency detection, without requiring modifications or fine-tuning of the base model. Extensive experiments across four mainstream open-source LLMs and multiple public benchmarks demonstrate that PADD significantly reduces attack success rates (0.4--3.0%) while maintaining low false rejection rates (0--5.2%). These results position PADD as a scalable, efficient, and practical safeguard for LLM safety.
论文检索
输入标题、作者或关键词，从 100,903 篇学术成果中精准定位
Advanced Persistent Threats (APTs) represent sophisticated cyberattacks that evade detection through stealthy, multistage operations, posing severe risks to critical infrastructure and organizational security. Due to their ability to effectively capture contextual information of attack behaviors, provenance graphs have emerged as a promising approach for APT detection. However, traditional binary edges in provenance graphs fail to represent the collaborative nature of APT attacks, where multiple entities coordinate in single operations, and local graph structures cannot capture the long-range dependencies across attack stages. To address these challenges, we propose HyperDetector, a novel hypergraph-based method for APT detection. First, we introduce hypergraph representation for provenance data, where hyperedges naturally connect multiple entities involved in system events, preserving the higher-order relational structures that characterize APT behaviors. Second, we employ block self-attention mechanisms that enable global reasoning across distant hypergraph regions, effectively linking dispersed attack indicators throughout the system. Through the synergistic integration of these approaches, HyperDetector achieves comprehensive understanding of both localized multi-entity collaborative behaviors and system-wide attack propagation patterns. Extensive evaluations across multiple prominent datasets demonstrate that HyperDetector outperforms state-of-the-art methods, showcasing its effectiveness for robust and holistic APT detection. Additionally, we make our code and datasets publicly available to facilitate reproducibility and foster further research in this critical area.
While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks. Adversaries exploit these vulnerabilities by poisoning documents provided by RAG system to manipulate LLM outputs. To counter this threat, we propose SecureCollaRAG, a Byzantine-tolerant collaborative RAG framework leveraging Multi-source Knowledge Validation Mechanism. Our approach enables agent system to securely verify document provenance through dynamic GNN-based credibility scoring, effectively preventing stealthy knowledge corruption attacks while preserving essential domain knowledge integrity. Through extensive evaluations and formal analysis, we demonstrate that SecureCollaRAG maintains robustness against attackers under non-IID data distributions. Content warning: This paper contains unfiltered content generated by LLMs that may contain malicious contents.
Large language models (LLMs) have been widely integrated into critical automated workflows, including contract review and job application processes. However, LLMs are susceptible to manipulation by fraudulent information, which can lead to harmful outcomes. Although advanced defense methods have been developed to address this issue, they often exhibit limitations in effectiveness, interpretability, and generalizability, particularly when applied to LLM-based applications. To address these challenges, we introduce FraudShield, a novel framework designed to protect LLMs from fraudulent content by leveraging a comprehensive analysis of fraud tactics. Specifically, FraudShield constructs and refines a fraud tactic–keyword knowledge graph to capture high-confidence associations between suspicious text and fraud techniques. The structured knowledge graph augments the original input by highlighting keywords and providing supporting evidence, guiding the LLM toward more secure responses. Extensive experiments show that FraudShield consistently outperforms state-of-the-art defenses across four mainstream LLMs and five representative fraud types, while also offering interpretable clues for the model's generations.
The InterPlanetary File System (IPFS) is the largest decentralized content-centric storage network. While its architecture enables resilient, distributed content delivery, it can be abused to host and disseminate malicious content. Public IPFS HTTP gateways further expand this threat surface, enabling attackers to deploy phishing websites and leverage gateway reputation to evade detection. This model can keep content available even after attackers go offline and challenges traditional phishing detection systems. We present a framework for monitoring and characterizing phishing on IPFS, leveraging a measurement platform that integrates multi-source data, including IPFS traffic and passive DNS. Over 11 months, we detect 10,489 phishing CIDs, grouped into 448 phishing clusters. 80% of detected CIDs originate from only 69 clustered campaigns indicating that targeting a small number of dominant clusters could yield high mitigation leverage. We also identify 588 gateways involved in dissemination, including 573 outside public gateway lists, and show that attackers can exploit caching across reputable gateways to amplify attacks and extend content availability. Finally, we find that traditional Web phishing countermeasures and IPFS blocklists provide insufficient protection. Our findings support practical mitigation and offer broader insights for trust and safety in decentralized web infrastructures.
Large Language Models (LLMs), as common components of modern web application backends and online services, are being widely deployed across various web infrastructures in the .gguf single-file format. This trend exposes their model parameter space to an unprecedented hardware attack surface, such as Bit-Flip attacks (BFA). This paper is the first to systematically discover and validate the existence of single-bit vulnerabilities in LLMs weight files: In the .gguf quantization format of mainstream open-source models (such as DeepSeek, QWEN), flipping a single bit can induce three types of targeted semantic-level faults, respectively-Artificial Flawed Intelligence (outputting factual errors), Artificial Weak Intelligence (catastrophic model failure), and Artificial Bad Intelligence (generating harmful content). By building an information-theoretic weight sensitivity entropy model and a probabilistic heuristic scanning framework called BitSifter, we achieved efficient localization of critical vulnerable bits in models with hundreds of millions of parameters. Furthermore, an end-to-end remote BFA chain was designed, enabling semantic-level attacks in real-world web server deployment scenarios: At an attack frequency of 464.3 times per second, the average time required for the first successful flip of the target bit is 31.7 seconds, without requiring high-cost equipment or complex prompt engineering. This study reveals a critical finding: under relatively modest remote-attack conditions, requiring only conventional network connectivity, flipping a single vulnerable bit within the tensor data segment can cause models deployed in web service environments to autonomously generate extremely malicious responses, such as ''humans should be exterminated'', or produce naturally fluent and difficult-to-detect erroneous replies to ordinary user queries. This demonstrates a pervasive and exploitable security vulnerability in LLMs systems at the fundamental hardware level.
Adapter-based Federated Large Language Models (FedLLMs) are widely adopted to reduce the computational, storage, and communication overhead of full-parameter fine-tuning for web-scale applications while preserving user privacy. By freezing the backbone and training only compact low-rank adapters, these methods appear to limit gradient leakage and thwart existing Gradient Inversion Attacks (GIAs). Contrary to this assumption, we show that low-rank adapters create new, exploitable leakage channels. We propose the Unordered-word-bag-based Text Reconstruction (UTR) attack, a novel GIA tailored to the unique structure of adapter-based FedLLMs. UTR overcomes three core challenges—low-dimensional gradients, frozen backbones, and combinatorially large reconstruction spaces—by: (i) inferring token presence from attention patterns in frozen layers, (ii) performing sentence-level inversion within the low-rank subspace of adapter gradients, and (iii) enforcing semantic coherence through constrained greedy decoding guided by language priors. Extensive experiments across diverse models (GPT2-Large, BERT, Qwen2.5-7B) and datasets (CoLA, SST-2, Rotten Tomatoes) demonstrate that UTR achieves near-perfect reconstruction accuracy (ROUGE-1/2 > 99), even with large batch sizes—settings where prior GIAs fail completely. Our results reveal a fundamental tension between parameter efficiency and privacy in FedLLMs, challenging the prevailing belief that lightweight adaptation inherently enhances security. Our code and data are available at https://github.com/shwksnshwowk-wq/GIA
Secure and undetectable transmission of visual intelligence through open networks presents a critical technological challenge in today's heavily monitored digital world. Text-based encryption is insufficient for massive, multimodal data from ubiquitous devices. The fundamental dilemma lies in transmitting large-scale visual data through noisy, adversarial channels while ensuring seamless concealment and robust recovery. In this paper, we bridge the gap by combining diffusion models with the universal presence and robustness of Quick Response (QR) codes for covert communication. Grounding steganographic security in the normative patterns of machine-readable symbols decouples it from the statistical properties of natural images. The reconstructions reside on the same semantic manifold as the originals, ensuring their functional equivalence in downstream tasks and maintaining robustness against quality degradation that would cripple pixel-based methods. Our training-free model compresses complex visual information into binary representations that integrate seamlessly into the data region of standard-compliant QR codes. This approach facilitates covert data transmission while preserving the full functionality and readability of the QR container. We achieve a compression ratio exceeding 400:1 while retaining semantic information for high-quality reconstruction. The preservation of all functional QR code components ensures complete compatibility with standard readers while concealing substantial visual content. Experimental results show a superior balance of security and performance compared to existing methods, all of which require no extra training.
Web tracking is increasingly pervasive, raising serious concerns about user privacy and security. Among existing techniques, pixel tracking is particularly stealthy and cost-effective, embedding invisible images that exfiltrate user activities to third-party servers. Current defenses, including filter list blocking and conventional machine learning, often fail to capture the cross-site associations that enable pixel tracking to evade detection. To address this limitation, we introduce TGNN, a framework that formulates pixel tracking detection as an edge classification task on a Tracking Directed Graph (TDG), which models third-party associations across websites. TGNN encodes HTTP traffic into structured quadruples and learns both semantic features and interaction patterns. To overcome the scarcity of reliable labels, we propose a large language model (LLM)-based annotation method that leverages minimal expert supervision to produce high-quality labels, significantly improving detection. Experiments conducted on traffic from the Alexa top-10K websites demonstrate that TGNN substantially outperforms existing baselines, while the LLM-based annotation achieves accuracy comparable to expert curation. Our large-scale measurement reveals that at least 16.74% of websites engage in pixel tracking via major third-party infrastructures, establishing cross-domain tracking as a pervasive practice in the wild and indicating a potential privacy threat in the modern Web ecosystem.
Federated knowledge graph embedding (FKGE) leverages distributed knowledge graphs (KGs) to collaboratively learn latent representations while preserving data privacy. However, this decentralized paradigm exposes vulnerabilities to client-side attacks, particularly untargeted poisoning attacks, which remain underexplored in FKGE and risk severe performance degradation. This paper presents the first systematic investigation of untargeted poisoning attacks on FKGE, unveiling their feasibility at both data and embedding levels. We design three novel poisoning attacks, among which the embedding-level attacks exhibit superior attack efficacy. To mitigate such threats, we further propose a learning-based defense mechanism that leverages embedding density for anomaly detection to identify malicious clients. Our theoretical results substantiate the effectiveness of both the attacks and the defense, and clarify why embedding-level poisoning is especially potent. Extensive experiments demonstrate that the proposed attacks induce significant performance deterioration, with core metrics dropping by up to 93%. Conversely, LDM effectively counters these attacks and restores performance to pre-attack levels or even surpasses them when mitigating embedding-level attacks.
Distributed Parameter-Efficient Fine-Tuning (PEFT) has emerged as a promising framework for personalizing Large Language Models (LLMs) by leveraging a collaboration between cloud servers and edge devices. However, this paradigm harbors a critical privacy vulnerability: the cloud can perform inversion attacks on the intermediate results (e.g., embeddings) sent from the edge to reconstruct the user's raw input prompt. Existing privacy-preserving techniques present a difficult trade-off: they are either too computationally expensive or they degrade model performance. To address this challenge, we introduce PrivSplit, a novel and lightweight protocol for prompt privacy in the Distributed PEFT framework. The core idea of PrivSplit is a ''Split and Compensate'' strategy: the edge device splits the input embedding into a public part sent to the cloud and a private part retained locally. The influence of this private part is then perfectly restored using a precise, lossless compensation mechanism during the interaction. We theoretically prove that PrivSplit is secure against both reconstruction and distinguishability attacks. Crucially, our method is perfectly lossless, ensuring that the model's performance is mathematically identical to the non-private baseline. This allows the security parameters to be arbitrarily strengthened without any impact on utility, breaking the conventional privacy-performance trade-off.
Secure manifold learning on datasets distributed among multiple data owners can benefit or even spawn many applications. For example, multiple service providers can jointly fit low-dimensional embeddings of their users' network behavior data to improve the accuracy of anomaly detection while addressing their privacy concerns about the datasets. In this paper, we focus on a classic manifold learning technique, known as isometric mapping (Isomap), and propose SIsomap, the first secure, distributed manifold learning system. We construct SIsomap based on secret sharing techniques and introduce careful optimizations. In particular, we propose two communication-efficient secure building blocks that focus on top-k and all-pairs shortest paths computation, respectively, and reduce secure operations by leveraging the characteristics of Isomap. Experimental results on both synthetic and real-world datasets demonstrate that our secure top-k and all-pairs shortest paths protocols are respectively up to 13.6× and 1818.5× faster than the state-of-the-art methods, and SIsomap as a whole is 11.1× to 28.8× faster than the baseline solution.
The deployment of self-hosted large language models (LLMs) has experienced unprecedented growth for enhanced data privacy and control. Yet, such deployment relies on diverse web services, whose vulnerabilities, although mentioned in a few studies, are largely underexplored, conflicting with the security tenet. From a systematic perspective, we propose LENS, a framework that explores and exploits vulnerabilities in self-hosted LLM services for comprehensive security evaluation. LENS integrates profiling and filtering, endpoint knowledge construction, and attack graph modeling for the automatic discovery, probing, and exploitation of public-facing LLM deployment targets, respectively. We conducted extensive empirical evaluation on real-world self-hosted LLM services across 16 mainstream platforms, 71,249 discovered deployment targets, and 307 API endpoints. Both quantitative and qualitative evidence reveal the prevalence of security vulnerabilities across different self-hosted LLM services. Notably, 75% of responsive targets allow web API interactions without authentication, rendering exploitation such as injection attacks (97% for Ollama), unauthenticated access (20.2% for AnythingLLM), and default credential abuse (60.6% for Dify). We have responsibly reported the findings to the relevant community and obtained 7 CVE IDs, including 4 critical vulnerabilities (CVSS > 9.0) and 2 high-severity ones.
The evaluation of Deep Research Agents is a critical challenge, as conventional outcome-based metrics fail to capture the nuances of their complex reasoning. Current evaluation faces two primary challenges: 1) a reliance on singular metrics like Pass@1, creating a ''high-score illusion'' that ignores the quality, efficiency, and soundness of the reasoning process; and 2) the failure of static benchmarks to quantify crucial attributes like robustness and latent capability. To address these gaps, we introduce TRACE (Trajectory-Aware Comprehensive Evaluation), a framework that holistically assesses the entire problem-solving trajectory. To counter the ''high-score illusion'', we propose a Hierarchical Trajectory Utility Function that quantifies process efficiency and cognitive quality, including evidence grounding, alongside accuracy. To measure deeper attributes, TRACE introduces a Scaffolded Capability Assessment protocol, quantifying an agent's latent ability by determining the minimum guidance needed for success. Our contributions include the TRACE framework, its novel metrics, and the accompanying DeepResearch-Bench with controllable complexity. Experiments show TRACE delivers a granular ranking that uncovers critical trade-offs between agent accuracy, efficiency, and robustness entirely missed by singular metrics.
Large Language Models (LLMs) have recently demonstrated remarkable reasoning abilities, yet hallucinate on knowledge-intensive tasks. Retrieval-augmented generation (RAG) mitigates this issue by grounding answers in external sources, e.g., knowledge graphs (KGs). However, existing KG-based RAG approaches rely on semantics-unaware path sampling and are weakly aligned with KG reasoning objectives, which limits further accuracy gains. They also feed retrieved paths directly into the reasoner without organizing them into answer-centered reasoning paths, hindering small LLMs' ability to leverage the retrieved knowledge. Furthermore, prior works predominantly rely on large LLMs (e.g., ChatGPT/GPT-4) or assume backbones above 7B parameters, leaving sub-7B models underexplored. We address this gap with RPO-RAG, the first KG-based RAG framework specifically designed for small LLMs, to the best of our knowledge. RPO-RAG introduces three key innovations: (1) a query-path semantic sampling strategy that provides informative supervisory signals; (2) a relation-aware preference optimization that aligns training with intermediate KG reasoning signals (e.g., relation); and (3) an answer-centered prompt design that organizes entities and reasoning paths in an interpretable format. Extensive experiments on two benchmark Knowledge Graph Question Answering (KGQA) datasets, WebQSP and CWQ, demonstrate that RPO-RAG effectively bridges the performance gap between small and large language models. On WebQSP, it improves F1 by up to 8.8%, reflecting enhanced answer precision, while on CWQ it achieves new state-of-the-art results among models under 8B parameters in both Hit and F1. Overall, RPO-RAG substantially improves the reasoning capability of small LLMs—even under 3B parameters—highlighting their potential for resource-efficient and practical on-device KGQA applications.
CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generation
Multimodal Retrieval-Augmented Generation (MRAG) enables Multimodal Large Language Models (MLLMs) to generate responses with external multimodal evidence. Recently, numerous video-based MRAG benchmarks have been proposed to evaluate model capabilities across retrieval and generation stages in MRAG. However, existing benchmarks remain limited in modality coverage and format diversity, often focusing on single- or limited-modality tasks, or coarse-grained scene understanding. To address these gaps, we introduce CFVBench, a large-scale, manually verified benchmark constructed from 599 publicly available videos, yielding 5,360 open-ended QA pairs. CFVBench, spans high-density formats and domains such as chart-heavy reports, news broadcasts, and software tutorials, requiring models to retrieve and reason over long temporal video spans while maintaining fine-grained multimodal information. Using CFVBench, we systematically evaluate 7 retrieval methods and 14 widely-used MLLMs, revealing a critical bottleneck: current models (even GPT5 or Gemini) struggle to capture transient yet essential fine-grained multimodal details. To mitigate this, we propose Adaptive Visual Refinement (AVR), a plug-and-paly framework that adaptively increases frame sampling density and selectively invokes external tools when necessary. Experiments show that AVR consistently enhances fine-grained multimodal comprehension and improves performance across all evaluated MLLMs.
The proliferation of e-commerce has made web shopping platforms key gateways for customers navigating the vast digital marketplace. Yet this rapid expansion has led to a noisy and fragmented information environment, increasing cognitive burden as shoppers explore and purchase products online. With promising potential to alleviate this challenge, agentic systems have garnered growing attention for automating user-side tasks in web shopping. Despite significant advancements, existing benchmarks fail to comprehensively evaluate how well agentic systems can curate products in open-web settings. Specifically, they have limited coverage of shopping scenarios, focusing only on simplified single-platform lookups rather than exploratory search. Moreover, they overlook personalization in evaluation, leaving unclear whether agents can adapt to diverse user preferences in realistic shopping contexts. To address this gap, we present AgenticShop, the first benchmark for evaluating agentic systems on personalized product curation in open-web environment. Crucially, our approach features realistic shopping scenarios, diverse user profiles, and a verifiable, checklist-driven personalization evaluation framework. Through extensive experiments, we demonstrate that current agentic systems remain largely insufficient, emphasizing the need for user-side systems that effectively curate tailored products across the modern web.
Retrieval-Augmented Generation (RAG) has emerged as a paradigm for enhancing large language models (LLMs) with external knowledge, yet existing graph-based methods face a fundamental limitation: entity-centric and chunk-centric approaches operate on representations anchored to original text without true knowledge fusion. While entity-centric methods connect logically related content and chunk-centric methods preserve context, both retrieve information separately through similarity search, missing emergent understanding from their synthesis. In this paper, we propose HyGRAG, a hierarchical graph RAG framework that transcends source documents by addressing three core challenges: constructing summaries that genuinely integrate contextual and relational information, leveraging these synthesized representations to access emergent knowledge during retrieval, and efficiently updating hierarchical structures for dynamic corpora. Specifically, we design hierarchical index structures over hybrid graphs with both chunk and entity nodes, then iteratively cluster them and generate LLM-based summaries. Then, we design context and relation-aware retrieval that searches across all abstraction levels while expanding through community membership. Moreover, we enable dynamic knowledge update through attachment-based algorithms with only local re-summarization. Experimental results show that HyGRAG improves the average accuracy of multi-hop reasoning tasks by 9.7%, while maintaining reasonable efficiency.
Graph-based Retrieval-Augmented Generation (RAG) typically operates on binary Knowledge Graphs (KGs). However, decomposing complex facts into binary triples often leads to semantic fragmentation and longer reasoning paths, increasing the risk of retrieval drift and computational overhead. In contrast, n-ary hypergraphs preserve high-order relational integrity, enabling shallower and more semantically cohesive inference. To exploit this topology, we propose HyperRAG, a framework tailored for n-ary hypergraphs featuring two complementary retrieval paradigms: (i) HyperRetriever learns structural-semantic reasoning over n-ary facts to construct query-conditioned relational chains. It enables accurate factual tracking, adaptive high-order traversal, and interpretable multi-hop reasoning under context constraints. (ii) HyperMemory leverages the LLM's parametric memory to guide beam search, dynamically scoring n-ary facts and entities for query-aware path expansion. Extensive evaluations on WikiTopics (11 closed-domain datasets) and three open-domain QA benchmarks (HotpotQA, MuSiQue, and 2WikiMultiHopQA) validate HyperRAG's effectiveness. HyperRetriever achieves the highest answer accuracy overall, with average gains of 2.95% in MRR and 1.23% in Hits@10 over the strongest baseline. Qualitative analysis further shows that HyperRetriever bridges reasoning gaps through adaptive and interpretable n-ary chain construction, benefiting both open and closed-domain QA. Our codes are publicly available at https://github.com/Vincent-Lien/HyperRAG.git.