With the rapid growth of multi-modal content on the Web, robust vision-language models are essential for semantic understanding and classification of web images under diverse and dynamic contexts, supporting Web applications such as multimedia search and recommendation. Prompt learning has proven effective for enhancing vision-language models in semantic image classification tasks. However, previous methods often suffer from poor generalization: the learned prompts tend to overfit the base classes seen during training, leading to poor performance on unseen classes and under distribution shifts. This issue is especially challenging in Web-scale data, where new classes emerge and distributions shift dynamically. To address these limitations, we propose PLIKD, a novel prompt learning method that integrates instance-aware knowledge distillation for robust Web-scale semantic image classification. Specifically, PLIKD introduces an instance-aware knowledge extraction module, which leverages multi-modal large language models through a step-by-step strategy to extract external knowledge for each image instance. To incorporate this extracted knowledge, PLIKD further introduces an instance-aware knowledge distillation module, which consists of two key steps: (1) a dual-teacher strategy for robust and informative knowledge distillation, and (2) fine-grained cross-modal alignment via Smooth and Sparse Optimal Transport. Extensive experiments demonstrate that PLIKD significantly improves generalization to both seen and unseen classes, and remains robust under distribution shifts, outperforming existing state-of-the-art methods on Web-scale semantic image classification.
论文检索
输入标题、作者或关键词，从 13,734 篇学术成果中精准定位
The multimodal Chinese idiom reading comprehension task aims to select the most appropriate idiom from a candidate list via the given text and image. This poses a significant challenge for the model to comprehend each Chinese idiom accurately. Existing multimodal Chinese idiom reading comprehension methods primarily focus on aligning contextual text and images, while overlooking two key attributes of Chinese idioms.(1) There is a discrepancy between the literal and metaphorical meanings of Chinese idioms. (2) The same Chinese idiom has different meanings in different scenarios, which requires targeted understanding by experts who specialize in different fields. To address the above challenges, we rethink the solution to the multimodal idiom reading comprehension task from a metaphorical perspective and propose a framework named MePe. Firstly, we propose a literal metaphorical semantic graph that systematically transforms the implicit discrepancy between the literal and metaphorical meanings of Chinese idioms into structured explicit relationships, thereby making metaphorical meanings more understandable. Then, we propose a mixture of idiom experts consisting of a literal idiom expert and a metaphorical idiom expert. Through division of labor and collaboration among these experts, we achieve an understanding of the dual meanings of Chinese idioms across different scenarios. Finally, we employ the maximum mean discrepancy to adjust the variance between the literal and metaphorical semantic features of Chinese idioms. By mapping these features onto a shared reproducing kernel Hilbert space, the model can better distinguish between the two based on contextual clues. Extensive experiments demonstrate that MePe achieves state-of-the-art performance on the MChIRC dataset.
Semantic understanding of popularity bias is a crucial yet underexplored challenge in recommender systems, where popular items are often favored at the expense of niche content. Most existing debiasing methods treat the semantic understanding of popularity bias as a matter of diversity enhancement or long-tail coverage, neglecting the deeper semantic layer that embodies the causal origins of the bias itself. Consequently, such shallow interpretations limit both their debiasing effectiveness and recommendation accuracy. In this paper, we propose FairLRM, a novel framework that bridges the gap in the semantic understanding of popularity bias with Recommendation via Large Language Model (RecLLM). FairLRM decomposes popularity bias into item-side and user-side components, using structured instruction-based prompts to enhance the model's comprehension of both global item distributions and individual user preferences. Unlike traditional methods that rely on surface-level features such as ''diversity'' or ''debiasing'', FairLRM improves the model's ability to semantically interpret and address the underlying bias. Through empirical evaluation, we demonstrate that FairLRM enhances fairness and recommendation accuracy through a trustworthy, semantically grounded treatment of popularity bias. The source code is shown in https://github.com/LuoRenqiang/FairLRM.
Temporal Knowledge Graph Completion (TKGC) aims to predict missing entities or relations based on historical facts, thereby facilitating the understanding of dynamic system evolution and supporting downstream reasoning tasks. However, existing methods predominantly focus on modeling sequential and structural dependencies, often overlooking the rich semantic information embedded in entities and relations, as well as the higher-order interactions among them, which limits their ability to handle complex, evolving scenarios effectively. To address these limitations, we propose DSTAG, a novel TKGC approach based on a semantic tag-enhanced dual-graph convolutional network. Our method leverages large language models to generate contextualized semantic multi-tags for both entities and relations (e.g., ''political event,'' ''economic activity''), thereby enriching their semantic representations. Furthermore, we introduce a semantic tag representation mechanism that captures higher-order dependencies during the aggregation and propagation of semantic tag information across graphs. DSTAG adopts a dual-graph convolutional network architecture, where the relation graph convolution extracts semantic features between temporal relationships and injects this information into the entity graph convolution, enabling joint modeling of entities and relations. We evaluate DSTAG on three widely used TKG benchmarks: ICEWS14, ICEWS18, and ICEWS05-15. Experimental results show that DSTAG achieves substantial MRR improvements over state-of-the-art baselines by 8.64%, 9.81% and 4.56%, respectively.
Video captioning aims to describe the content of a given video with condensed natural language sentences. Such a captioning task is full of challenges since the high requirements for visual-textual relevance and multimodal fusion understanding. Previous works primarily focus on visual content modeling, often overlooking the rich semantic correlations between visual and textual modalities, which results in incomplete understanding of the multimodal context and suboptimal caption accuracy. In this paper, we propose a multimodal graph conditioned diffusion model for video captioning, named MGCDVc. The idea behind our model is to incorporate graph-based relational reasoning with diffusion-based generative modeling to jointly model cross-modal relationships and capture latent semantic structure. Specifically, we learn a set of latent concept anchors to bridge the visual and textual modality nodes, enabling the construction of a weighted multimodal graph. Then we introduce the graph conditioned diffusion strategy which generates the textual semantic nodes and associated edges under the graph structure awareness condition. Furthermore, a soft pruning mechanism is designed to filter out low-quality nodes, thus further refining the generated multimodal graph to provide more accurate semantic structural guidance for caption generation. Experimental results on several popular datasets demonstrate that our model achieves better performance in video captioning task.
Phishing is a threat in which attackers masquerade as legitimate entities to steal sensitive data. While understanding the phishing ecosystem is critical for developing effective countermeasures, prior research has largely studied phishing through post-attack data, with limited examination of the attacker's perspective and how phishing campaigns are built. Critically, the Deep and Dark Web (D2Web) serves as the primary marketplace and knowledge-sharing platform where attackers acquire phishing tools (e.g., phishing kits), exchange techniques, and trade compromised credentials. Analyzing D2Web forums therefore provides unique visibility into the supply chain of phishing attacks pre-deployment, enabling proactive understanding of emerging threats and attack methodologies. This study addresses this gap through a comprehensive analysis of 394,034 posts (343,334 unique) collected from 13 D2Web forums spanning 2013 to 2025, from which 70,055 phishing-related posts are identified. We employ a LLM-based approach to efficiently extract key information, including phishing attack components (e.g., credentials, phishing pages, SMTP servers), targeted services (e.g., PayPal, Netflix), and component authors. This extracted data is mapped to a seven-stage attack scenario framework derived from empirical case studies. Our analysis reveals longitudinal trends in component availability, target service distribution, post type evolution, and the most active contributors annually, while characterizing pricing dynamics across different attack components. The results provide the first attacker-centric, macro-level longitudinal analysis of the phishing ecosystem, offering insights into how phishing infrastructure and markets have evolved over more than a decade. These findings contribute to a deeper understanding of the phishing supply chain and inform more effective detection and prevention strategies.
Network protocols are the foundation of modern communication, yet their implementations often contain semantic vulnerabilities stemming from inadequate understanding of specification semantics. Existing gray-box and black-box testing approaches lack semantic modeling of protocols, making it difficult to precisely express testing intent and cover boundary conditions. Moreover, they typically rely on coarse-grained oracles such as crashes, which are inadequate for identifying deep semantic vulnerabilities. To address these limitations, we present a semantics-aware fuzzing framework, SemFuzz. The framework leverages large language models to extract structured semantic rules from RFC documents and generates test cases that intentionally violate these rules to encode specific testing intents. It then detects deep semantic vulnerabilities by comparing the observed responses with the expected ones. Evaluation on seven widely deployed protocol implementations shows that SemFuzz identified sixteen potential vulnerabilities, ten of which have been confirmed. Among the confirmed vulnerabilities, five were previously unknown and four have been assigned CVEs. These results demonstrate the effectiveness of SemFuzz in detecting semantic vulnerabilities.
In the literature, prior research on Security-oriented Video Understanding (SVU) has predominantly focused on detecting and locating the threats (e.g., shootings, robberies) in videos, while largely lacking the effective capability to generate and evaluate the threat causes. Motivated by these gaps, this paper introduces a new chat paradigm SVU task, i.e., In-depth Security-oriented Video Understanding (DeepSVU), which aims to not only identify and locate the threats but also attribute and evaluate the causes of threatening segments in detail. Furthermore, this paper reveals two key challenges in the proposed task: 1) how to effectively model the coarse-to-fine physical-world information (e.g., human behavior, object interactions and background context) to boost the DeepSVU task, and 2) how to adaptively trade off these factors. Addressing these challenges is crucial for improving VAD, especially for identifying, locating, and attributing anomalies. To tackle these challenges, this paper proposes a new Unified Physical-world Regularized MoE (UPRM) approach. Specifically, UPRM incorporates two key components: the Unified Physical-world Enhanced MoE (UPE) Block and the Physical-world Trade-off Regularizer (PTR), to address the above two challenges, respectively. Extensive experiments conduct on our DeepSVU instructions datasets (i.e., UCF-C instructions and CUVA instructions) demonstrate that UPRM outperforms several advanced Video-LLMs as well as non-LLM approaches. such information.These justify the importance of the coarse-to-fine physical-world information in the DeepSVU task and demonstrate the effectiveness of our UPRM in capturing such information.
The real-time demand for web security makes Security Information and Event Management (SIEM) platforms and their applied security rule an integral part of the intrusion detection life-cycle. However, the heterogeneity of vendor-specific rules (e.g., Splunk SPL, Microsoft KQL, IBM AQL, Google YARA-L, and RSA ESA) makes cross-platform rule reuse extremely difficult, requiring deep domain knowledge for reliable conversion. As a result, an autonomous and accurate rule conversion framework can significantly lead to effort savings, preserving the value of existing rules. In this paper, we propose ARuleCon, an agentic SIEM-rule conversion approach. Using ARuleCon, the security professionals do not need to distill the source rules' logic and re-map it to target vendors, instead, they provide the source rules, the documentation of the target rules and ARuleCon can purposely convert to the target vendors without more intervention. To achieve this, ARuleCon is equipped with intermediate representation (IR) that aligns core detection logic into vendor-neutral layer, agentic RAG pipeline that retrieves authoritative official vendor documentation to address the convension/schema mismatches, and Python-based consistency check that running both source and target rules in controlled test environments to mitigate subtle semantic drifts. We present a comprehensive evaluation of ARuleCon ranging from textual alignment between the source and target rules, and the execution success of target rules, showcasing ARuleCon can convert rules with higher fidelity, outperforming the baseline LLM models by 15% averagely. Finally, we perform a case study and interview with our industry collaborators 1, which showcases that ARuleCon can significantly save the expert's time on understanding the cross-SIEM's documentation and remapping the logic.
Hidden web interfaces in Internet of Things (IoT) devices pose significant security threats by unintentionally exposing inadequately protected functionalities, enabling attackers to bypass authentication, alter configurations, leak sensitive data, or execute arbitrary commands. Despite recent advancements, current detection approaches suffer from two critical challenges: 1) inadequately model the complex internal routing mechanisms of IoT firmware, leading to incomplete interface enumeration and substantial false negatives; and 2) inefficiently generate probing requests and verify unauthorized access due to limited semantic understanding of interface communication protocols. To overcome these challenges, we introduce FalconScope, a novel system combining precise firmware routing modeling and Large Language Model (LLM)-driven semantic analysis to detect hidden web interfaces effectively and efficiently. FalconScope achieves this through two key innovations: 1) a static analysis technique precisely reconstructs the device's internal routing mechanisms, enabling comprehensive enumeration of Routing Unique Identifiers and their corresponding backend handlers; and 2) an LLM-powered semantic engine automatically generates syntactically and semantically valid HTTP requests to efficiently trigger backend logic, coupled with semantic validation of device responses to accurately confirm unauthorized access. Evaluations on 11 real-world IoT devices from four major vendors demonstrate that FalconScope significantly surpasses existing state-of-the-art tools, detecting 620 hidden web interfaces—103 times more than IoTScope—while consuming only 3.6% of its analysis time. Following responsible disclosure, 50 issues have been assigned CVE IDs.
The Domain Name System (DNS) plays a crucial role in modern web applications; however, manipulations such as hijacking, tampering, and censorship can disrupt domain resolution, posing significant privacy and security risks. While such manipulations are prevalent across global DNS infrastructures, their scope and mechanisms remain poorly understood. Existing studies focus on country-level censorship or rely on authoritative data and passive traffic from selected domains, which prevents a comprehensive understanding. Moreover, the dynamic nature of modern DNS resolution, in which a single domain may resolve to thousands of edge servers, further complicates the detection of manipulated responses. In this work, we propose a novel approach for measuring DNS manipulations based on resolution path analysis. Our method leverages CNAME chains and attributes of intermediate nodes in the DNS resolution process to link dynamic resolution results, enabling accurate detection of manipulation in highly dynamic DNS environments. We conduct large-scale measurements for 2,283 popular domains across global open DNS infrastructures. Measurement results reveal critical insights into DNS manipulation, uncovering the strategies and preferences of malicious manipulation operators and demonstrating how specific domains are exploited.
Graph unlearning (GU), which removes nodes, edges, or features from trained graph neural networks (GNNs), is crucial in Web applications where graph data may contain sensitive, mislabeled, or malicious information. However, existing GU methods lack a clear understanding of the key factors that determine unlearning effectiveness, leading to three fundamental limitations: (1) impractical and inaccurate GU difficulty assessment due to test-access requirements and invalid assumptions, (2) ineffectiveness on hard-to-unlearn tasks, and (3) misaligned evaluation protocols that overemphasize easy tasks and fail to capture true forgetting capability. To address these issues, we establish GNN memorization as a new perspective for understanding graph unlearning and propose MGU, a Memorization-guided Graph Unlearning framework. MGU achieves three key advances: it provides accurate and practical difficulty assessment across different GU tasks, develops an adaptive strategy that dynamically adjusts unlearning objectives based on difficulty levels, and establishes a comprehensive evaluation protocol that aligns with practical requirements. Extensive experiments on ten real-world graphs demonstrate that MGU consistently outperforms state-of-the-art baselines in forgetting quality, computational efficiency, and utility preservation.
Graph Neural Networks (GNNs) have become a pivotal framework for modeling graph-structured data, enabling a wide range of applications from social network analysis to molecular chemistry. By integrating large language models (LLMs), text-attributed graphs (TAGs) enhance node representations with rich textual semantics, significantly boosting the expressive power of graph-based learning. However, this synergy introduces critical vulnerabilities in both topology and text. Although specialized attack methods have been designed for each of these aspects, no work has yet unified them into a comprehensive approach. In this work, we propose the Interpretable Multi-Dimensional Graph Attack (IMDGA), a human-centric framework orchestrating multi-level perturbations across graph structure and textual features. IMDGA utilizes three tightly integrated modules to craft attacks that balance interpretability and impact, enabling a deeper understanding of Graph-LLM vulnerabilities. Through rigorous theoretical analysis and comprehensive empirical evaluations on diverse datasets and architectures, IMDGA demonstrates superior interpretability, attack effectiveness, stealthiness, and robustness compared to existing methods. By exposing these underexplored semantic vulnerabilities, our work offers valuable insights for improving Graph-LLM resilience. Our code is available at https://github.com/bwfan-bit/IMDGA.
The proliferation of face forgery content on the Web poses a severe threat to online trust, social media security, and the credibility of digital information. Existing detection approaches often fail to generalize across diverse forgery types and unseen scenarios commonly encountered in web-scale applications. Recent studies have utilized visual large language models (VLMs) to answer not only ''Is this face a forgery?'' but also ''Why is the face a forgery?'' These studies introduced forgery-related attributes, such as forgery location and type, to construct deepfake VQA datasets and train VLMs, achieving high accuracy while providing human-understandable explanatory text descriptions. However, these methods still have limitations. For example, they do not fully leverage face quality-related attributes, which are often abnormal in forged faces, and they lack effective training strategies for forgery-aware VLMs. In this paper, we extend the VQA dataset to create DD-VQA+, which features a richer set of attributes and a more diverse range of samples. Furthermore, we introduce a novel forgery detection framework, MGFFD-VLM, which integrates an Attribute-Driven Hybrid LoRA Strategy to enhance the capabilities of Visual Large Language Models (VLMs). Additionally, our framework incorporates Multi-Granularity Prompt Learning and a Forgery-Aware Training Strategy. By transforming classification and forgery segmentation results into prompts, our method not only improves forgery classification but also enhances interpretability. To further boost detection performance, we design multiple forgery-related auxiliary losses. Experimental results demonstrate that our approach surpasses existing methods in both text-based forgery judgment and analysis, achieving superior accuracy.
Large Language Models (LLMs) have achieved remarkable progress in understanding and generation tasks, yet they remain highly susceptible to adversarial prompt attacks that bypass safety safeguards and induce the generation of harmful content. Existing pre-generation and post-generation defense methods typically rely on intent recognition, surface-level features, or external classifiers, rendering them vulnerable to evasion via subtle prompt perturbations while incurring substantial computational overhead. In this paper, we propose PADD (Prefix-based Attention Divergence Detector), a lightweight pre-generation defense mechanism that leverages internal model signals for robust attack detection. At its core, PADD prepends a lightweight safety prefix to the input prompt and compares attention distributions between the original and prefixed prompts. By transforming cross-prompt comparisons into self-comparisons via composite signals of attention divergence and attention plasticity, PADD achieves strong separability between adversarial and benign prompts with low-latency detection, without requiring modifications or fine-tuning of the base model. Extensive experiments across four mainstream open-source LLMs and multiple public benchmarks demonstrate that PADD significantly reduces attack success rates (0.4--3.0%) while maintaining low false rejection rates (0--5.2%). These results position PADD as a scalable, efficient, and practical safeguard for LLM safety.
Advanced Persistent Threats (APTs) represent sophisticated cyberattacks that evade detection through stealthy, multistage operations, posing severe risks to critical infrastructure and organizational security. Due to their ability to effectively capture contextual information of attack behaviors, provenance graphs have emerged as a promising approach for APT detection. However, traditional binary edges in provenance graphs fail to represent the collaborative nature of APT attacks, where multiple entities coordinate in single operations, and local graph structures cannot capture the long-range dependencies across attack stages. To address these challenges, we propose HyperDetector, a novel hypergraph-based method for APT detection. First, we introduce hypergraph representation for provenance data, where hyperedges naturally connect multiple entities involved in system events, preserving the higher-order relational structures that characterize APT behaviors. Second, we employ block self-attention mechanisms that enable global reasoning across distant hypergraph regions, effectively linking dispersed attack indicators throughout the system. Through the synergistic integration of these approaches, HyperDetector achieves comprehensive understanding of both localized multi-entity collaborative behaviors and system-wide attack propagation patterns. Extensive evaluations across multiple prominent datasets demonstrate that HyperDetector outperforms state-of-the-art methods, showcasing its effectiveness for robust and holistic APT detection. Additionally, we make our code and datasets publicly available to facilitate reproducibility and foster further research in this critical area.
CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generation
Multimodal Retrieval-Augmented Generation (MRAG) enables Multimodal Large Language Models (MLLMs) to generate responses with external multimodal evidence. Recently, numerous video-based MRAG benchmarks have been proposed to evaluate model capabilities across retrieval and generation stages in MRAG. However, existing benchmarks remain limited in modality coverage and format diversity, often focusing on single- or limited-modality tasks, or coarse-grained scene understanding. To address these gaps, we introduce CFVBench, a large-scale, manually verified benchmark constructed from 599 publicly available videos, yielding 5,360 open-ended QA pairs. CFVBench, spans high-density formats and domains such as chart-heavy reports, news broadcasts, and software tutorials, requiring models to retrieve and reason over long temporal video spans while maintaining fine-grained multimodal information. Using CFVBench, we systematically evaluate 7 retrieval methods and 14 widely-used MLLMs, revealing a critical bottleneck: current models (even GPT5 or Gemini) struggle to capture transient yet essential fine-grained multimodal details. To mitigate this, we propose Adaptive Visual Refinement (AVR), a plug-and-paly framework that adaptively increases frame sampling density and selectively invokes external tools when necessary. Experiments show that AVR consistently enhances fine-grained multimodal comprehension and improves performance across all evaluated MLLMs.
Retrieval-Augmented Generation (RAG) has emerged as a paradigm for enhancing large language models (LLMs) with external knowledge, yet existing graph-based methods face a fundamental limitation: entity-centric and chunk-centric approaches operate on representations anchored to original text without true knowledge fusion. While entity-centric methods connect logically related content and chunk-centric methods preserve context, both retrieve information separately through similarity search, missing emergent understanding from their synthesis. In this paper, we propose HyGRAG, a hierarchical graph RAG framework that transcends source documents by addressing three core challenges: constructing summaries that genuinely integrate contextual and relational information, leveraging these synthesized representations to access emergent knowledge during retrieval, and efficiently updating hierarchical structures for dynamic corpora. Specifically, we design hierarchical index structures over hybrid graphs with both chunk and entity nodes, then iteratively cluster them and generate LLM-based summaries. Then, we design context and relation-aware retrieval that searches across all abstraction levels while expanding through community membership. Moreover, we enable dynamic knowledge update through attachment-based algorithms with only local re-summarization. Experimental results show that HyGRAG improves the average accuracy of multi-hop reasoning tasks by 9.7%, while maintaining reasonable efficiency.
SketchMind: Understanding Abstract Sketches with MLLMs for Fine-Grained Sketch-Based Image Retrieval
Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) aims to retrieve images that accurately correspond to abstract hand-drawn sketches, requiring the model to understand sparse and abstract visual cues. Existing methods tend to rely on convolutional networks or metric learning to align sketch and image features, often overlooking the inherent abstraction and semantic ambiguity present in sketches. This limitation results in an insufficient understanding of fine-grained visual details. To address this challenge, we propose SketchMind, a novel method that leverages Multi-modal Large Language Models (MLLMs) to enhance abstract sketch understanding in FG-SBIR. Specifically, we use MLLMs to generate auxiliary textual descriptions based on the given sketches via a Visual Question Answering (VQA) strategy. To effectively incorporate these descriptions, we construct a graph structure with the sketch as the central node and the generated texts as peripheral nodes. A graph attention scheme is employed to perform uncertainty-aware feature fusion, enabling the model to suppress noisy or irrelevant textual information. Furthermore, to enhance both inter- and intra-modal fine-grained alignment, we design a Multi-scale Cross-modal Jigsaw Matching module in combination with a self-supervised learning strategy, which captures local and global visual correspondences across modalities more effectively. Extensive experiments on three benchmark FG-SBIR datasets demonstrate that SketchMind achieves superior performance over existing state-of-the-art methods, proving its effectiveness. Code is available at https://github.com/li1changxing/MLLM_FG_SBIR/.