Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly initialised backbones. However, there is limited discussion in the community for the reasons that pseudo-alignment occurs. In this work, we conduct a thorough investigation into the root causes of pseudo-alignment in LLM4TS and build a connection of pseudo-alignment to the cone effect in LLM. We demonstrate that pseudo-alignment arises from the interplay of cone effect within pretrained LLM components and the intrinsically low-dimensional manifold of time-series data. In addition, we also introduce TimeSUP, a novel technique designed to mitigate this issue and improve forecast performance in existing LLM4TS approaches. TimeSUP addresses this by increasing the time series manifold to more closely match the intrinsic dimension of language embeddings, allowing the model to distinguish temporal signals clearly while still capturing shared structures across modalities. As a result, representations for time and language tokens remain distinct yet exhibit high cosine similarity, signifying that the model preserves each modality's unique features while learning their commonalities in a unified embedding space. Empirically, TimeSUP consistently outperforms state-of-the-art LLM4TS methods and other lightweight baselines on long-term forecasting performance. Furthermore, it can be seamlessly integrated into four existing LLM4TS pipelines and delivers significant improvements in forecasting performance.
论文检索
输入标题、作者或关键词，从 954 篇学术成果中精准定位
Web-scale platforms and online services rely on log-based anomaly detection to safeguard availability, latency SLOs, and user experience. In real-world web interactions, system logs often exhibit irregular temporal intervals, bursty densities, and heterogeneous semantics, which pose significant challenges for log anomaly detection. Existing methods such as LSTM and Transformer assume a fixed input window, which conflicts with the inherently irregular nature of system logs. Moreover, most prior works build a single-view representation, overlooking the multi-relational nature of logs. To overcome these challenges, we propose DyLogNet, a dynamic multi-relational graph framework for log anomaly detection. Specifically, this framework constructs a density-aware dynamic graph with variable-length windows, and represents logs from three relational perspectives: temporal co-occurrence, semantic similarity, and anomaly tendency. Next, we design a cross-layer attention mechanism that integrates heterogeneous structures to highlight the most relevant relations and enhance event representations. Furthermore, a cross-snapshot memory injection module updates global memory through a recurrent unit and injects it into current graph representations via an affine transformation, enabling temporal continuity. Experiments on three public log datasets demonstrate that DyLogNet outperforms state-of-the-art methods, especially in few-shot scenarios.
Multi-domain knowledge graph completion (MKGC) aims to improve missing triple prediction in a target KG by transferring knowledge from other support KGs. Existing methods typically enforce consistency constraints on equivalent entities across KGs to transfer knowledge, which risks suppressing domain-specific contextual information of entities. This design can also compromise entity representation information from all KG domains, impeding performance improvements, especially in low-resource data scenarios. To address this, we pioneer a generation-based paradigm for MKGC and propose DMKGC, a conditional diffusion-guided knowledge transfer framework. Our key insight is to treat each KG as a partial view of the entity entire information, and generate informative domain-general entity embeddings through diffusion models conditioned on support KGs. Particularly, we first initialize domain-agnostic entity embeddings as prior entity embeddings, and then encode them within individual KGs. Afterward, we fuse equivalent entities from support KGs as the conditional diffusion generation guidance. We leverage the prior entity embeddings as the proxy generation objective, which ensures this conditional generation to be unbiased towards any conditioned KGs. Simultaneously, we also train the generated embeddings to be predictive across KGs, thus preserving domain-specific information. Extensive experiments on 14 KGs in 3 benchmarks demonstrate a 4.3% average MRR improvement in tail entity prediction over state-of-the-art methods, with sustained gains in low-resource data settings.
Revealing the underlying causal mechanisms in the real world is crucial for scientific and technological progress. Despite notable advances in recent decades, the lack of high-quality data and the reliance of traditional causal discovery algorithms (TCDA) on the assumption of no latent confounders, as well as their tendency to overlook the precise semantics of latent variables, have long been major obstacles to the broader application of causal discovery. To address this issue, we propose a novel causal modeling framework, TLVD, which integrates the metadata-based reasoning capabilities of large language models (LLMs) with the data-driven modeling capabilities of TCDA for inferring latent variables and their semantics. Specifically, we first employ a data-driven approach to construct a causal graph that incorporates latent variables. Then, we employ multi-LLM collaboration for latent variable inference, modeling this process as a game with incomplete information and seeking its Bayesian Nash Equilibrium (BNE) to infer the possible specific latent variables. Finally, to validate the inferred latent variables across multiple real-world web-based data sources, we leverage LLMs for evidence exploration to ensure traceability. We comprehensively evaluate TLVD on three de-identified real patient datasets provided by a hospital and two benchmark datasets. Extensive experimental results confirm the effectiveness and reliability of TLVD, with average improvements of 32.67% in Acc, 62.21% in CAcc, and 26.72% in ECit across the five datasets.
CausalSKyHop: Knowledge-Aware Causal Explanation of Dynamic GNNs via Higher-Order Semantic Reasoning
Dynamic Graph Neural Networks (DyGNNs) are widely used to model web-scale semantic-rich graph data (e.g., social networks, knowledge graphs), but their inability to explain predictions ground-ed in structured knowledge remains a challenge, especially when predictions rely on complex higher-order substructures. We propose CausalSKyHop, a semantic- and knowledge-aware framework that explains DyGNNs by uncovering causal higher-order patterns in evolving knowledge structures. To model the semantic fabric of the graph, CausalSKyHop incorporates a Higher-Order Structural Causal Model to capture multi-node knowledge dependencies, and uses contrastive learning to isolate semantically-meaningful causal relationships from spurious ones. A dynamic correlation module further separates persistent knowledge from evolving semantic contexts. Through knowledge-infused, structure-aware variational graph autoencoders, our method produces interpretable causal subgraphs that capture the dynamic flow of knowledge and semantics. Experimental evaluations on multiple web and knowledge-rich graph benchmarks demonstrate that CausalSKyHop consistently outperforms state-of-the-art explainable DyGNNs, achieving notable improvements in both explanation fidelity and downstream prediction accuracy. A detailed case study further illustrates how our method uncovers stable, semantically coherent causal pathways—in contrast to the fragmented explanations of baseline methods—providing intuitive evidence for its superior interpretability. This work establishes the critical role of explicit semantic and knowledge integration through higher-order causal reasoning for building transparent and trustworthy DyGNNs on the web.
The explosive growth of multimodal web data demands communication that transmits meaning rather than raw bits. Existing semantic-communication systems often fail under noise, missing modalities, and distribution shifts because they optimize surface features instead of modality-invariant knowledge. We present Grasp, a knowledge-centric framework for cross-modal communication. Grasp segments streams into semantic blocks and builds a graph over them; a lightweight Graph Neural Networks (GNN) produces schedulable, importance-weighted representations. At its core is knowledge purification : we minimize a conditional mutual information upper bound to perform a three-way disentanglement—strongly related, weakly related, and task-irrelevant components—so that only essential semantics are transmitted while non-essential factors are suppressed. To maintain synchrony, we introduce one-to-two temporal contrastive learning to achieve triple alignment of video, audio, and text despite sampling asynchrony. For efficient transmission, Grasp uses a cross-modal shared vector-quantization codebook—a discrete knowledge codebook —updated by multimodal attention. At the receiver, a soft-recovery mechanism leverages this shared knowledge to robustly reconstruct semantics under low signal-to-noise ratio (SNR) or missing modalities, yielding graceful degradation. Across web tasks—including cross-modal retrieval and missing-modality inference—Grasp improves knowledge consistency, semantic fidelity, and downstream performance over strong baselines while maintaining low latency. These results show that communication structured around purified knowledge is key to building robust, semantic-aware systems for the modern web.
Text classification is a crucial and fundamental task in web content mining. Compared with the previous learning paradigm of pre-training and fine-tuning by cross entropy loss, the recently proposed supervised contrastive learning approach has received tremendous attention due to its powerful feature learning capability and robustness. Although several studies have incorporated this technique for text classification, some limitations remain. First, many text datasets are imbalanced, and the learning mechanism of supervised contrastive learning is sensitive to data imbalance, which may harm the model's performance. Moreover, these models leverage separate classification branches with cross entropy and supervised contrastive learning branches without explicit mutual guidance. To this end, we propose a novel model named SharpReCL for imbalanced text classification tasks. First, we obtain the prototype vector of each class in the balanced classification branch to act as a representation of each class. Then, by further explicitly leveraging the prototype vectors, we construct a proper and sufficient target sample set with the same size for each class to perform the supervised contrastive learning procedure. The empirical results show the effectiveness of our model, which even outperforms popular large language models across several datasets. Our code is available https://github.com/KEAML-JLU/SharpReCL
As knowledge and semantics on the web grow increasingly complex, enhancing Large Language Models (LLMs)' comprehension and reasoning capabilities has become particularly important. Chain-of-Thought (CoT) prompting has been shown to enhance the reasoning capabilities of LLMs. However, it still falls short on logical reasoning tasks that rely on symbolic expressions and strict deductive rules. Neuro-symbolic methods address this gap by enforcing formal correctness through external solvers. Yet these solvers are highly format-sensitive, and small instabilities in model outputs can lead to frequent processing failures. The LLM-driven approaches avoid parsing brittleness, but they lack structured representations and process-level error-correction mechanisms. To further enhance the logical reasoning capabilities of LLMs, we propose MatrixCoT, a structured CoT framework with a matrix-based plan. Specifically, we normalize and type natural language expressions and attach explicit citation fields, and introduce a matrix-based planning method to preserve global relations among steps. The plan thus becomes a verifiable artifact and execution becomes more stable. For verification, we also add a feedback-driven replanning mechanism. Under semantic-equivalence constraints, it identifies omissions and defects, rewrites and compresses the dependency matrix, and produces a more trustworthy final answer. Experiments on five logical-reasoning benchmarks and five LLMs show that, without relying on external solvers, MatrixCoT enhances both the robustness and interpretability of LLMs when tackling complex symbolic reasoning tasks, while maintaining competitive performance.
Multi-modal knowledge graphs (MMKGs) enrich traditional knowledge graphs by incorporating heterogeneous modalities such as textual descriptions and visual content, offering complementary semantic cues for knowledge reasoning. However, existing approaches often overlook the structural dependencies within each modality, apply static or coarse-grained fusion strategies, and insufficiently model relational semantics. We propose a Multi-Granularity Multi-Modal Knowledge Graph Representation Learning Method via Subgraph-aware Adaptive Fusion and Hierarchical Relation Modeling (SAFER ), which implement multi-modal knowledge representation through adaptive fusion of multi-granularity information such as multi-modal semantics, knowledge structures and relations. SAFER explicitly constructs modality-specific subgraphs and employs structure-aware graph attention networks to effectively capture intra-modal structural dependencies. We propose an adaptive multi-modal fusion mechanism, which aggregates modality-specific embeddings at the semantic level by dynamically assigning entity-specific modality weights. We design a two-stage multi-granularity knowledge relation modeling strategy, which utilizes a structure-aware multi-modal adaptive pre-fusion to preserve topological information and a relation-aware graph attention network (RGAT) post-fusion to encode relational semantics. Extensive experiments on several benchmark datasets demonstrate that the proposed SAFER significantly outperforms competitive baselines on link prediction and relation reasoning tasks.
Knowledge Graph Question Answering (KGQA) aims to answer natural language questions by reasoning over structured knowledge graphs (KGs). While large language models (LLMs) have advanced KGQA through their strong reasoning capabilities, existing methods continue to struggle to fully exploit both the rich knowledge encoded in KGs and the reasoning capabilities of LLMs, particularly in complex scenarios. They often assume complete KG coverage and lack mechanisms to judge when external information is needed, and their reasoning remains locally myopic, failing to maintain coherent multi-step planning, leading to reasoning failures even when relevant knowledge exists. We propose Graph-RFT, a novel two-stage reinforcement fine-tuning KGQA framework with a ''plan–KGsearch–and–Websearch–during–think'' paradigm, that enables LLMs to perform autonomous planning and adaptive retrieval scheduling across KG and web sources under incomplete knowledge conditions. Graph-RFT introduces a chain-of-thought (CoT) fine-tuning method with a customized plan–retrieval dataset activates structured reasoning and resolves the GRPO cold-start problem. It then introduces a novel plan–retrieval guided reinforcement learning process integrates explicit planning and retrieval actions with a multi-reward design, enabling coverage-aware retrieval scheduling. It employs a Cartesian-inspired planning module to decompose complex questions into ordered sub-questions, and logical expression to guide tool invocation for globally consistent multi-step reasoning. This reasoning–retrieval process is optimized with a multi-reward combining outcome and retrieval-specific signals, enabling the model to learn when and how to combine KG and web retrieval effectively. Experiments on multiple KGQA benchmarks demonstrate that Graph-RFT achieves superior performance over strong baselines, even with smaller LLM backbones, and substantially improves complex question decomposition, factual coverage, and tool coordination.
Semantic understanding of popularity bias is a crucial yet underexplored challenge in recommender systems, where popular items are often favored at the expense of niche content. Most existing debiasing methods treat the semantic understanding of popularity bias as a matter of diversity enhancement or long-tail coverage, neglecting the deeper semantic layer that embodies the causal origins of the bias itself. Consequently, such shallow interpretations limit both their debiasing effectiveness and recommendation accuracy. In this paper, we propose FairLRM, a novel framework that bridges the gap in the semantic understanding of popularity bias with Recommendation via Large Language Model (RecLLM). FairLRM decomposes popularity bias into item-side and user-side components, using structured instruction-based prompts to enhance the model's comprehension of both global item distributions and individual user preferences. Unlike traditional methods that rely on surface-level features such as ''diversity'' or ''debiasing'', FairLRM improves the model's ability to semantically interpret and address the underlying bias. Through empirical evaluation, we demonstrate that FairLRM enhances fairness and recommendation accuracy through a trustworthy, semantically grounded treatment of popularity bias. The source code is shown in https://github.com/LuoRenqiang/FairLRM.
Temporal Knowledge Graph Completion (TKGC) aims to predict missing entities or relations based on historical facts, thereby facilitating the understanding of dynamic system evolution and supporting downstream reasoning tasks. However, existing methods predominantly focus on modeling sequential and structural dependencies, often overlooking the rich semantic information embedded in entities and relations, as well as the higher-order interactions among them, which limits their ability to handle complex, evolving scenarios effectively. To address these limitations, we propose DSTAG, a novel TKGC approach based on a semantic tag-enhanced dual-graph convolutional network. Our method leverages large language models to generate contextualized semantic multi-tags for both entities and relations (e.g., ''political event,'' ''economic activity''), thereby enriching their semantic representations. Furthermore, we introduce a semantic tag representation mechanism that captures higher-order dependencies during the aggregation and propagation of semantic tag information across graphs. DSTAG adopts a dual-graph convolutional network architecture, where the relation graph convolution extracts semantic features between temporal relationships and injects this information into the entity graph convolution, enabling joint modeling of entities and relations. We evaluate DSTAG on three widely used TKG benchmarks: ICEWS14, ICEWS18, and ICEWS05-15. Experimental results show that DSTAG achieves substantial MRR improvements over state-of-the-art baselines by 8.64%, 9.81% and 4.56%, respectively.
Logical reasoning with large language models (LLMs) has received growing attention. One mainstream approach translates natural language into formal logic and then applies symbolic solvers for deduction. While effective in many tasks, these LLM-based translators often fail to generate consistent symbolic representations when the same concept appears in different linguistic forms. Such inconsistencies break logical coherence and lead to solver errors. However, most existing benchmarks lack this type of linguistic variation, which frequently occurs in real-world text, leaving the problem underexplored. To address this gap, we present SoLT, a benchmark that systematically rewrites reasoning datasets into diverse yet logically equivalent forms across multiple levels. Beyond evaluation, SoLT also provides a general method to enrich any dataset with linguistic diversity while preserving both meaning and logic. To further enhance the stability of LLM-based reasoning, we propose MenTaL, which explicitly guides models to build a concept–symbol mapping table during translation. By linking equivalent expressions to shared symbols, MenTaL maintains consistency and mitigates symbol drift. Experiments on SoLT demonstrate that LLMs indeed suffer from inconsistent symbol mapping under linguistic variation, leading to significant drops in reasoning accuracy. Meanwhile, applying MenTaL brings clear and stable performance improvements across diverse inputs. Overall, our findings reveal that overlooking linguistic diversity hides key weaknesses in LLM-based translators, and our work offers a step toward more reliable logical reasoning in varied real-world scenarios. Our code is available at https://github.com/wufeiwuwoshihua/LinguDiver.
While Large Language Models (LLMs) are increasingly deployed in Web applications such as search, dialogue, and recommendation systems, their reliance on large-scale Web data raises serious privacy concerns, particularly the risk of memorizing sensitive content. Existing Membership Inference Attacks (MIA) rely heavily on the surface form of inputs, rendering them ineffective against semantically preserved but structurally altered samples. This methodological weakness results in widespread false negatives and compromises the integrity of privacy evaluations in large-scale Web corpora. To address this limitation, we propose Adversarial Semantic Membership Inference Attack (ASMIA). ASMIA enhances MIA effectiveness by generating semantically diverse adversarial samples, extracting multi-layer attention features from the target model, and training a contrastive classifier that leverages similarity metrics and logarithmic probabilities to distinguish members from non-members. Experiments on LLMs trained with Wikipedia, a representative large-scale Web corpus, demonstrate that ASMIA significantly outperforms existing methods, highlighting the value of semantic perturbations and attention patterns in detecting training data leakage.
Social event detection (SED) aims to detect events (news) on social media platforms, which is essential in various applications, such as public opinion surveillance, disaster control, and market monitoring. However, social media platforms are characterized by the generation of short, dynamic, and multi-source social messages. This makes annotating social messages time-consuming and difficult (label scarcity), and poses a challenge to the widespread application of SED. Despite efforts, existing unsupervised SED models rely on graph structures to address the lack of textual content, resulting in unstable performance in dynamic social messages. To solve the above challenges, this work proposes an unsupervised SED framework with an LLM enhancement and Heterogeneous Graph induction (LHG). Specifically, to address the label scarcity problem, LHG generates pseudo-labels for initial social messages through an LLM. Considering the unreliable nature of LLM-generated labels, LHG designed a Meta-Path Guided Label Similarity Selector (MPLSS). In detail, MPLSS in LHG calculates the similarity of these pseudo-labels and constructs the initial social messages corresponding to the pseudo-labels into triplets based on the meta-paths in the heterogeneous information graph (HIG), thereby mitigating problems caused by LLMs, such as hallucination. Afterward, to improve stability, LHG not only utilizes the structural information in HIG via MPLSS, but also reduces the embedding distortion of the hierarchical structure in HIG and sentences via a hyperbolic representation, thereby ensuring that there is sufficient available information in dynamic social messages. Extensive experiments show that LHG achieves state-of-the-art (SOTA) results on two widely used real-world datasets.
Deductive and abductive reasoning are two critical paradigms for analyzing knowledge graphs, enabling applications from financial query answering to scientific discovery. Deductive reasoning on knowledge graphs usually involves retrieving entities that satisfy a complex logical query, while abductive reasoning generates plausible logical hypotheses from observations. Despite their clear synergistic potential, where deduction can validate hypotheses and abduction can uncover deeper logical patterns, existing methods address them in isolation. To bridge this gap, we propose DARK, a unified framework for Deductive and Abductive Reasoning in Knowledge graphs. As a masked diffusion model capable of capturing the bidirectional relationship between queries and conclusions, DARK has two key innovations. First, to better leverage deduction for hypothesis refinement during abductive reasoning, we introduce a self-reflective denoising process that iteratively generates and validates candidate hypotheses against the observed conclusion. Second, to discover richer logical associations, we propose a logic-exploration reinforcement learning approach that simultaneously masks queries and conclusions, enabling the model to explore novel reasoning compositions. Extensive experiments on multiple benchmark knowledge graphs show that DARK achieves competitive performance on both deductive and abductive reasoning tasks, demonstrating the significant benefits of our unified approach.
Entity alignment (EA) is to identify equivalent entities across different knowledge graphs (KGs), which can help fuse these KGs into a more comprehensive one. Previous EA methods mainly focus on aligning a pair of KGs, and to the best of our knowledge, no existing EA method considers aligning multiple (more than two) KGs. To fill this research gap, in this work, we study a novel problem of aligning multiple KGs and propose an effective framework named MultiEA to solve the problem. First, we embed the entities of all the candidate KGs into a common feature space by a shared KG encoder. Then, we explore three alignment strategies to minimize the distances among pre-aligned entities. In particular, we propose an innovative inference enhancement technique to improve the alignment performance by incorporating high-order similarities. Finally, to verify the effectiveness of MultiEA, we construct two new real-world benchmark datasets and conduct extensive experiments on them. The results show that our MultiEA can effectively and efficiently align multiple KGs in a single pass. We release the source code at: https://github.com/wzCSDN/MultiEA.
Long-term memory is critical for dialogue systems that support continuous, sustainable, and personalized interactions. However, existing methods rely on continuous summarization or OpenIE-based graph construction paired with fixed Top- k retrieval, leading to limited adaptability across query categories and high computational overhead. In this paper, we propose HingeMem, a boundary-guided long-term memory that operationalizes event segmentation theory to build an interpretable indexing interface via boundary-triggered hyperedges over four elements: person, time, location, and topic. When any such element changes, HingeMem draws a boundary and writes the current segment, thereby reducing redundant operations and preserving salient context. To enable robust and efficient retrieval under diverse information needs, HingeMem introduces query-adaptive retrieval mechanisms that jointly decide (a) what to retrieve: determine the query-conditioned routing over the element-indexed memory; (b) how much to retrieve: control the retrieval depth based on the estimated query type. Extensive experiments across LLM scales (from 0.6B to production-tier models; e.g., Qwen3-0.6B to Qwen-Flash) on LOCOMO show that HingeMem achieves approximately 20% relative improvement over strong baselines without query categories specification, while reducing computational cost (68%\downarrow question answering token cost compared to HippoRAG2). Beyond advancing memory modeling, HingeMem's adaptive retrieval makes it a strong fit for web applications requiring efficient and trustworthy memory over extended interactions.
Video captioning aims to describe the content of a given video with condensed natural language sentences. Such a captioning task is full of challenges since the high requirements for visual-textual relevance and multimodal fusion understanding. Previous works primarily focus on visual content modeling, often overlooking the rich semantic correlations between visual and textual modalities, which results in incomplete understanding of the multimodal context and suboptimal caption accuracy. In this paper, we propose a multimodal graph conditioned diffusion model for video captioning, named MGCDVc. The idea behind our model is to incorporate graph-based relational reasoning with diffusion-based generative modeling to jointly model cross-modal relationships and capture latent semantic structure. Specifically, we learn a set of latent concept anchors to bridge the visual and textual modality nodes, enabling the construction of a weighted multimodal graph. Then we introduce the graph conditioned diffusion strategy which generates the textual semantic nodes and associated edges under the graph structure awareness condition. Furthermore, a soft pruning mechanism is designed to filter out low-quality nodes, thus further refining the generated multimodal graph to provide more accurate semantic structural guidance for caption generation. Experimental results on several popular datasets demonstrate that our model achieves better performance in video captioning task.
Open-source ecosystems such as NPM and PyPI are increasingly targeted by supply chain attacks, yet existing detection methods either depend on fragile handcrafted rules or data-driven features that fail to capture evolving attack semantics. We present IntelGuard, a retrieval-augmented generation (RAG) based framework that integrates expert analytical reasoning into automated malicious package detection. IntelGuard constructs a structured knowledge base from over 8,000 threat intelligence reports, linking malicious code snippets with behavioral descriptions and expert reasoning. When analyzing new packages, it retrieves semantically similar malicious examples and applies LLM-guided reasoning to assess whether code behaviors align with intended functionality. Experiments on 4,027 real-world packages show that IntelGuard achieves 99% accuracy and a 0.50% false positive rate, while maintaining 96.5% accuracy on obfuscated code. Deployed on PyPI.org, it discovered 54 previously unreported malicious packages, demonstrating interpretable and robust detection guided by expert knowledge.