Large Language Models (LLMs), as common components of modern web application backends and online services, are being widely deployed across various web infrastructures in the .gguf single-file format. This trend exposes their model parameter space to an unprecedented hardware attack surface, such as Bit-Flip attacks (BFA). This paper is the first to systematically discover and validate the existence of single-bit vulnerabilities in LLMs weight files: In the .gguf quantization format of mainstream open-source models (such as DeepSeek, QWEN), flipping a single bit can induce three types of targeted semantic-level faults, respectively-Artificial Flawed Intelligence (outputting factual errors), Artificial Weak Intelligence (catastrophic model failure), and Artificial Bad Intelligence (generating harmful content). By building an information-theoretic weight sensitivity entropy model and a probabilistic heuristic scanning framework called BitSifter, we achieved efficient localization of critical vulnerable bits in models with hundreds of millions of parameters. Furthermore, an end-to-end remote BFA chain was designed, enabling semantic-level attacks in real-world web server deployment scenarios: At an attack frequency of 464.3 times per second, the average time required for the first successful flip of the target bit is 31.7 seconds, without requiring high-cost equipment or complex prompt engineering. This study reveals a critical finding: under relatively modest remote-attack conditions, requiring only conventional network connectivity, flipping a single vulnerable bit within the tensor data segment can cause models deployed in web service environments to autonomously generate extremely malicious responses, such as ''humans should be exterminated'', or produce naturally fluent and difficult-to-detect erroneous replies to ordinary user queries. This demonstrates a pervasive and exploitable security vulnerability in LLMs systems at the fundamental hardware level.
论文检索
输入标题、作者或关键词，从 954 篇学术成果中精准定位
Adapter-based Federated Large Language Models (FedLLMs) are widely adopted to reduce the computational, storage, and communication overhead of full-parameter fine-tuning for web-scale applications while preserving user privacy. By freezing the backbone and training only compact low-rank adapters, these methods appear to limit gradient leakage and thwart existing Gradient Inversion Attacks (GIAs). Contrary to this assumption, we show that low-rank adapters create new, exploitable leakage channels. We propose the Unordered-word-bag-based Text Reconstruction (UTR) attack, a novel GIA tailored to the unique structure of adapter-based FedLLMs. UTR overcomes three core challenges—low-dimensional gradients, frozen backbones, and combinatorially large reconstruction spaces—by: (i) inferring token presence from attention patterns in frozen layers, (ii) performing sentence-level inversion within the low-rank subspace of adapter gradients, and (iii) enforcing semantic coherence through constrained greedy decoding guided by language priors. Extensive experiments across diverse models (GPT2-Large, BERT, Qwen2.5-7B) and datasets (CoLA, SST-2, Rotten Tomatoes) demonstrate that UTR achieves near-perfect reconstruction accuracy (ROUGE-1/2 > 99), even with large batch sizes—settings where prior GIAs fail completely. Our results reveal a fundamental tension between parameter efficiency and privacy in FedLLMs, challenging the prevailing belief that lightweight adaptation inherently enhances security. Our code and data are available at https://github.com/shwksnshwowk-wq/GIA
Secure and undetectable transmission of visual intelligence through open networks presents a critical technological challenge in today's heavily monitored digital world. Text-based encryption is insufficient for massive, multimodal data from ubiquitous devices. The fundamental dilemma lies in transmitting large-scale visual data through noisy, adversarial channels while ensuring seamless concealment and robust recovery. In this paper, we bridge the gap by combining diffusion models with the universal presence and robustness of Quick Response (QR) codes for covert communication. Grounding steganographic security in the normative patterns of machine-readable symbols decouples it from the statistical properties of natural images. The reconstructions reside on the same semantic manifold as the originals, ensuring their functional equivalence in downstream tasks and maintaining robustness against quality degradation that would cripple pixel-based methods. Our training-free model compresses complex visual information into binary representations that integrate seamlessly into the data region of standard-compliant QR codes. This approach facilitates covert data transmission while preserving the full functionality and readability of the QR container. We achieve a compression ratio exceeding 400:1 while retaining semantic information for high-quality reconstruction. The preservation of all functional QR code components ensures complete compatibility with standard readers while concealing substantial visual content. Experimental results show a superior balance of security and performance compared to existing methods, all of which require no extra training.
Web tracking is increasingly pervasive, raising serious concerns about user privacy and security. Among existing techniques, pixel tracking is particularly stealthy and cost-effective, embedding invisible images that exfiltrate user activities to third-party servers. Current defenses, including filter list blocking and conventional machine learning, often fail to capture the cross-site associations that enable pixel tracking to evade detection. To address this limitation, we introduce TGNN, a framework that formulates pixel tracking detection as an edge classification task on a Tracking Directed Graph (TDG), which models third-party associations across websites. TGNN encodes HTTP traffic into structured quadruples and learns both semantic features and interaction patterns. To overcome the scarcity of reliable labels, we propose a large language model (LLM)-based annotation method that leverages minimal expert supervision to produce high-quality labels, significantly improving detection. Experiments conducted on traffic from the Alexa top-10K websites demonstrate that TGNN substantially outperforms existing baselines, while the LLM-based annotation achieves accuracy comparable to expert curation. Our large-scale measurement reveals that at least 16.74% of websites engage in pixel tracking via major third-party infrastructures, establishing cross-domain tracking as a pervasive practice in the wild and indicating a potential privacy threat in the modern Web ecosystem.
Federated knowledge graph embedding (FKGE) leverages distributed knowledge graphs (KGs) to collaboratively learn latent representations while preserving data privacy. However, this decentralized paradigm exposes vulnerabilities to client-side attacks, particularly untargeted poisoning attacks, which remain underexplored in FKGE and risk severe performance degradation. This paper presents the first systematic investigation of untargeted poisoning attacks on FKGE, unveiling their feasibility at both data and embedding levels. We design three novel poisoning attacks, among which the embedding-level attacks exhibit superior attack efficacy. To mitigate such threats, we further propose a learning-based defense mechanism that leverages embedding density for anomaly detection to identify malicious clients. Our theoretical results substantiate the effectiveness of both the attacks and the defense, and clarify why embedding-level poisoning is especially potent. Extensive experiments demonstrate that the proposed attacks induce significant performance deterioration, with core metrics dropping by up to 93%. Conversely, LDM effectively counters these attacks and restores performance to pre-attack levels or even surpasses them when mitigating embedding-level attacks.
Distributed Parameter-Efficient Fine-Tuning (PEFT) has emerged as a promising framework for personalizing Large Language Models (LLMs) by leveraging a collaboration between cloud servers and edge devices. However, this paradigm harbors a critical privacy vulnerability: the cloud can perform inversion attacks on the intermediate results (e.g., embeddings) sent from the edge to reconstruct the user's raw input prompt. Existing privacy-preserving techniques present a difficult trade-off: they are either too computationally expensive or they degrade model performance. To address this challenge, we introduce PrivSplit, a novel and lightweight protocol for prompt privacy in the Distributed PEFT framework. The core idea of PrivSplit is a ''Split and Compensate'' strategy: the edge device splits the input embedding into a public part sent to the cloud and a private part retained locally. The influence of this private part is then perfectly restored using a precise, lossless compensation mechanism during the interaction. We theoretically prove that PrivSplit is secure against both reconstruction and distinguishability attacks. Crucially, our method is perfectly lossless, ensuring that the model's performance is mathematically identical to the non-private baseline. This allows the security parameters to be arbitrarily strengthened without any impact on utility, breaking the conventional privacy-performance trade-off.
Secure manifold learning on datasets distributed among multiple data owners can benefit or even spawn many applications. For example, multiple service providers can jointly fit low-dimensional embeddings of their users' network behavior data to improve the accuracy of anomaly detection while addressing their privacy concerns about the datasets. In this paper, we focus on a classic manifold learning technique, known as isometric mapping (Isomap), and propose SIsomap, the first secure, distributed manifold learning system. We construct SIsomap based on secret sharing techniques and introduce careful optimizations. In particular, we propose two communication-efficient secure building blocks that focus on top-k and all-pairs shortest paths computation, respectively, and reduce secure operations by leveraging the characteristics of Isomap. Experimental results on both synthetic and real-world datasets demonstrate that our secure top-k and all-pairs shortest paths protocols are respectively up to 13.6× and 1818.5× faster than the state-of-the-art methods, and SIsomap as a whole is 11.1× to 28.8× faster than the baseline solution.
The deployment of self-hosted large language models (LLMs) has experienced unprecedented growth for enhanced data privacy and control. Yet, such deployment relies on diverse web services, whose vulnerabilities, although mentioned in a few studies, are largely underexplored, conflicting with the security tenet. From a systematic perspective, we propose LENS, a framework that explores and exploits vulnerabilities in self-hosted LLM services for comprehensive security evaluation. LENS integrates profiling and filtering, endpoint knowledge construction, and attack graph modeling for the automatic discovery, probing, and exploitation of public-facing LLM deployment targets, respectively. We conducted extensive empirical evaluation on real-world self-hosted LLM services across 16 mainstream platforms, 71,249 discovered deployment targets, and 307 API endpoints. Both quantitative and qualitative evidence reveal the prevalence of security vulnerabilities across different self-hosted LLM services. Notably, 75% of responsive targets allow web API interactions without authentication, rendering exploitation such as injection attacks (97% for Ollama), unauthenticated access (20.2% for AnythingLLM), and default credential abuse (60.6% for Dify). We have responsibly reported the findings to the relevant community and obtained 7 CVE IDs, including 4 critical vulnerabilities (CVSS > 9.0) and 2 high-severity ones.
The evaluation of Deep Research Agents is a critical challenge, as conventional outcome-based metrics fail to capture the nuances of their complex reasoning. Current evaluation faces two primary challenges: 1) a reliance on singular metrics like Pass@1, creating a ''high-score illusion'' that ignores the quality, efficiency, and soundness of the reasoning process; and 2) the failure of static benchmarks to quantify crucial attributes like robustness and latent capability. To address these gaps, we introduce TRACE (Trajectory-Aware Comprehensive Evaluation), a framework that holistically assesses the entire problem-solving trajectory. To counter the ''high-score illusion'', we propose a Hierarchical Trajectory Utility Function that quantifies process efficiency and cognitive quality, including evidence grounding, alongside accuracy. To measure deeper attributes, TRACE introduces a Scaffolded Capability Assessment protocol, quantifying an agent's latent ability by determining the minimum guidance needed for success. Our contributions include the TRACE framework, its novel metrics, and the accompanying DeepResearch-Bench with controllable complexity. Experiments show TRACE delivers a granular ranking that uncovers critical trade-offs between agent accuracy, efficiency, and robustness entirely missed by singular metrics.
Large Language Models (LLMs) have recently demonstrated remarkable reasoning abilities, yet hallucinate on knowledge-intensive tasks. Retrieval-augmented generation (RAG) mitigates this issue by grounding answers in external sources, e.g., knowledge graphs (KGs). However, existing KG-based RAG approaches rely on semantics-unaware path sampling and are weakly aligned with KG reasoning objectives, which limits further accuracy gains. They also feed retrieved paths directly into the reasoner without organizing them into answer-centered reasoning paths, hindering small LLMs' ability to leverage the retrieved knowledge. Furthermore, prior works predominantly rely on large LLMs (e.g., ChatGPT/GPT-4) or assume backbones above 7B parameters, leaving sub-7B models underexplored. We address this gap with RPO-RAG, the first KG-based RAG framework specifically designed for small LLMs, to the best of our knowledge. RPO-RAG introduces three key innovations: (1) a query-path semantic sampling strategy that provides informative supervisory signals; (2) a relation-aware preference optimization that aligns training with intermediate KG reasoning signals (e.g., relation); and (3) an answer-centered prompt design that organizes entities and reasoning paths in an interpretable format. Extensive experiments on two benchmark Knowledge Graph Question Answering (KGQA) datasets, WebQSP and CWQ, demonstrate that RPO-RAG effectively bridges the performance gap between small and large language models. On WebQSP, it improves F1 by up to 8.8%, reflecting enhanced answer precision, while on CWQ it achieves new state-of-the-art results among models under 8B parameters in both Hit and F1. Overall, RPO-RAG substantially improves the reasoning capability of small LLMs—even under 3B parameters—highlighting their potential for resource-efficient and practical on-device KGQA applications.
CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generation
Multimodal Retrieval-Augmented Generation (MRAG) enables Multimodal Large Language Models (MLLMs) to generate responses with external multimodal evidence. Recently, numerous video-based MRAG benchmarks have been proposed to evaluate model capabilities across retrieval and generation stages in MRAG. However, existing benchmarks remain limited in modality coverage and format diversity, often focusing on single- or limited-modality tasks, or coarse-grained scene understanding. To address these gaps, we introduce CFVBench, a large-scale, manually verified benchmark constructed from 599 publicly available videos, yielding 5,360 open-ended QA pairs. CFVBench, spans high-density formats and domains such as chart-heavy reports, news broadcasts, and software tutorials, requiring models to retrieve and reason over long temporal video spans while maintaining fine-grained multimodal information. Using CFVBench, we systematically evaluate 7 retrieval methods and 14 widely-used MLLMs, revealing a critical bottleneck: current models (even GPT5 or Gemini) struggle to capture transient yet essential fine-grained multimodal details. To mitigate this, we propose Adaptive Visual Refinement (AVR), a plug-and-paly framework that adaptively increases frame sampling density and selectively invokes external tools when necessary. Experiments show that AVR consistently enhances fine-grained multimodal comprehension and improves performance across all evaluated MLLMs.
The proliferation of e-commerce has made web shopping platforms key gateways for customers navigating the vast digital marketplace. Yet this rapid expansion has led to a noisy and fragmented information environment, increasing cognitive burden as shoppers explore and purchase products online. With promising potential to alleviate this challenge, agentic systems have garnered growing attention for automating user-side tasks in web shopping. Despite significant advancements, existing benchmarks fail to comprehensively evaluate how well agentic systems can curate products in open-web settings. Specifically, they have limited coverage of shopping scenarios, focusing only on simplified single-platform lookups rather than exploratory search. Moreover, they overlook personalization in evaluation, leaving unclear whether agents can adapt to diverse user preferences in realistic shopping contexts. To address this gap, we present AgenticShop, the first benchmark for evaluating agentic systems on personalized product curation in open-web environment. Crucially, our approach features realistic shopping scenarios, diverse user profiles, and a verifiable, checklist-driven personalization evaluation framework. Through extensive experiments, we demonstrate that current agentic systems remain largely insufficient, emphasizing the need for user-side systems that effectively curate tailored products across the modern web.
Retrieval-Augmented Generation (RAG) has emerged as a paradigm for enhancing large language models (LLMs) with external knowledge, yet existing graph-based methods face a fundamental limitation: entity-centric and chunk-centric approaches operate on representations anchored to original text without true knowledge fusion. While entity-centric methods connect logically related content and chunk-centric methods preserve context, both retrieve information separately through similarity search, missing emergent understanding from their synthesis. In this paper, we propose HyGRAG, a hierarchical graph RAG framework that transcends source documents by addressing three core challenges: constructing summaries that genuinely integrate contextual and relational information, leveraging these synthesized representations to access emergent knowledge during retrieval, and efficiently updating hierarchical structures for dynamic corpora. Specifically, we design hierarchical index structures over hybrid graphs with both chunk and entity nodes, then iteratively cluster them and generate LLM-based summaries. Then, we design context and relation-aware retrieval that searches across all abstraction levels while expanding through community membership. Moreover, we enable dynamic knowledge update through attachment-based algorithms with only local re-summarization. Experimental results show that HyGRAG improves the average accuracy of multi-hop reasoning tasks by 9.7%, while maintaining reasonable efficiency.
Graph-based Retrieval-Augmented Generation (RAG) typically operates on binary Knowledge Graphs (KGs). However, decomposing complex facts into binary triples often leads to semantic fragmentation and longer reasoning paths, increasing the risk of retrieval drift and computational overhead. In contrast, n-ary hypergraphs preserve high-order relational integrity, enabling shallower and more semantically cohesive inference. To exploit this topology, we propose HyperRAG, a framework tailored for n-ary hypergraphs featuring two complementary retrieval paradigms: (i) HyperRetriever learns structural-semantic reasoning over n-ary facts to construct query-conditioned relational chains. It enables accurate factual tracking, adaptive high-order traversal, and interpretable multi-hop reasoning under context constraints. (ii) HyperMemory leverages the LLM's parametric memory to guide beam search, dynamically scoring n-ary facts and entities for query-aware path expansion. Extensive evaluations on WikiTopics (11 closed-domain datasets) and three open-domain QA benchmarks (HotpotQA, MuSiQue, and 2WikiMultiHopQA) validate HyperRAG's effectiveness. HyperRetriever achieves the highest answer accuracy overall, with average gains of 2.95% in MRR and 1.23% in Hits@10 over the strongest baseline. Qualitative analysis further shows that HyperRetriever bridges reasoning gaps through adaptive and interpretable n-ary chain construction, benefiting both open and closed-domain QA. Our codes are publicly available at https://github.com/Vincent-Lien/HyperRAG.git.
SketchMind: Understanding Abstract Sketches with MLLMs for Fine-Grained Sketch-Based Image Retrieval
Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) aims to retrieve images that accurately correspond to abstract hand-drawn sketches, requiring the model to understand sparse and abstract visual cues. Existing methods tend to rely on convolutional networks or metric learning to align sketch and image features, often overlooking the inherent abstraction and semantic ambiguity present in sketches. This limitation results in an insufficient understanding of fine-grained visual details. To address this challenge, we propose SketchMind, a novel method that leverages Multi-modal Large Language Models (MLLMs) to enhance abstract sketch understanding in FG-SBIR. Specifically, we use MLLMs to generate auxiliary textual descriptions based on the given sketches via a Visual Question Answering (VQA) strategy. To effectively incorporate these descriptions, we construct a graph structure with the sketch as the central node and the generated texts as peripheral nodes. A graph attention scheme is employed to perform uncertainty-aware feature fusion, enabling the model to suppress noisy or irrelevant textual information. Furthermore, to enhance both inter- and intra-modal fine-grained alignment, we design a Multi-scale Cross-modal Jigsaw Matching module in combination with a self-supervised learning strategy, which captures local and global visual correspondences across modalities more effectively. Extensive experiments on three benchmark FG-SBIR datasets demonstrate that SketchMind achieves superior performance over existing state-of-the-art methods, proving its effectiveness. Code is available at https://github.com/li1changxing/MLLM_FG_SBIR/.
Retrieval-Augmented Generation (RAG) effectively grounds Large Language Models (LLMs) with external knowledge and is widely applied to Web-related tasks. However, its scalability is hindered by excessive context length and redundant retrievals. Recent research on soft context compression aims to address this by encoding long documents into compact embeddings, yet they often underperform non-compressed RAG due to their reliance on auto-encoder-like full-compression that forces the encoder to compress all document information regardless of relevance to the input query. In this work, we conduct an analysis on this paradigm and reveal two fundamental limitations: (I) Infeasibility: full-compression conflicts with the LLM's downstream generation behavior; and (II) Non-necessity: full-compression is unnecessary and dilutes task-relevant information density. Motivated by these insights, we introduce SeleCom, a selector-based soft compression framework for RAG that redefines the encoder's role as query-conditioned information selector. The selector is decoder-only and is trained with a massive, diverse and difficulty-graded synthetic QA dataset with curriculum learning. Extensive experiments show that SeleCom significantly outperforms existing soft compression approaches and achieves competitive or superior performance to non-compression baselines, while reducing computation and latency by 33.8%~84.6%.
Reasoning models have gained significant attention due to their strong performance, particularly when enhanced with retrieval augmentation. However, these models often incur high computational costs, as both retrieval and reasoning tokens contribute substantially to the overall resource usage. In this work, we make the following contributions: (1) we propose a retrieval-augmented reasoning model that dynamically adjusts the length of the retrieved document list based on the query and retrieval results; (2) we develop a cost-aware advantage function for training of efficient retrieval-augmented reasoning models through reinforcement learning; and (3) we explore both memory- and latency-bound implementations of the proposed cost-aware framework for both proximal and group relative policy optimization algorithms. We evaluate our approach on seven public question answering datasets and demonstrate significant efficiency gains, without compromising effectiveness. In fact, we observed that the model latency decreases by ~16-20% across datasets, while its effectiveness increases by ~5% on average, in terms of exact match.
Medical question answering (QA) demands reliable access to rich, domain-specific knowledge. A prevalent strategy enhances large language models (LLMs) with either external knowledge retrieved from medical corpora or parametric knowledge embedded with model parameters. Existing methods generally fall into two paradigms: Retrieval-Augmented Generation (RAG), which grounds reasoning in retrieved evidence, and Generation-Augmented Generation (GAG), which relies exclusively on internally generated contexts. However, RAG often suffers from noisy or incomplete retrieval, whereas GAG is prone to hallucinated or inaccurate content due to unconstrained generation. Both limitations can misguide reasoning and compromise answer reliability. To overcome these challenges, we propose MedRGAG, a unified retrieval--generation augmented framework that seamlessly integrates external and parametric knowledge for medical QA. MedRGAG consists of two core modules: Knowledge-Guided Context Completion (KGCC), which steers the generator to produce background documents that complement knowledge gaps revealed by retrieval, and Knowledge-Aware Document Selection (KADS), which adaptively selects an optimal combination of retrieved and generated documents to construct concise yet comprehensive evidence for answers. Extensive experiments on five medical QA benchmarks demonstrate that MedRGAG achieves a 12.5% improvement over MedRAG and a 4.5% gain over MedGENIE, highlighting the effectiveness of unifying retrieval and generation for knowledge-intensive medical reasoning. Further analyses show that our framework effectively generates complementary contexts and recovers informative retrieved evidence. Our code and data are publicly available at https://github.com/ll0ruc/MedRGAG.
Reasoning over structured graphs remains a fundamental challenge for Large Language Models (LLMs), particularly when scaling to large graphs. Existing approaches typically follow the retrieval-augmented generation (RAG) paradigm: first retrieving subgraphs relevant to the query and then generating answers conditioned on the retrieved subgraphs. However, such two-phase pipelines often struggle to faithfully incorporate graph structure, since the generation process is ultimately constrained by the quality and completeness of the retrieved subgraph. Although many advanced retrievers have been proposed recently to mitigate this issue, they are usually tailored to the training graphs and generalize poorly to unseen graphs, which limits their practical applicability. In this work, we propose Reasoning by Exploration (RoE), a novel approach that unifies retrieval and generation by framing reasoning over graphs as a process of graph exploration. At each step, the LLM selects candidate nodes and edges to explore, gradually constructing reasoning paths and generating answers along the way. To enable effective exploration, RoE is trained in two stages: supervised fine-tuning (SFT) on gold reasoning paths, followed by reinforcement learning (RL) to enhance exploration effectiveness and generalization. Experiments on benchmark datasets demonstrate that RoE achieves substantial overall improvements over baselines, while also generalizing effectively to unseen graphs. The code can be found at https://github.com/haoyuhan1/RoE.
Scientific document retrieval is a critical task for enabling knowledge discovery and supporting research across diverse domains. However, existing dense retrieval methods often struggle to capture fine-grained scientific concepts in texts due to their reliance on holistic embeddings and limited domain understanding. Recent approaches leverage large language models (LLMs) to extract fine-grained semantic entities and enhance semantic matching, but they typically treat entities as independent fragments, overlooking the multi-faceted nature of scientific concepts. To address this limitation, we propose Pairwise Semantic Matching (PairSem), a framework that represents relevant semantics as entity–aspect pairs, capturing complex, multi-faceted scientific concepts. PairSem is unsupervised, base retriever-agnostic, and plug-and-play, enabling precise and context-aware matching without requiring query-document labels or entity annotations. Extensive experiments on multiple datasets and retrievers demonstrate that PairSem significantly improves retrieval performance, highlighting the importance of modeling multi-aspect semantics in scientific information retrieval.