Real-world multimodal knowledge graphs (MKGs) are inherently heterogeneous, modeling entities that are associated with diverse modalities. Traditional knowledge graph embedding (KGE) methods excel at learning continuous representations of entities and relations, yet they are typically designed for unimodal settings. Recent approaches extend KGE to multimodal settings but remain constrained, often processing modalities in isolation, resulting in weak cross-modal alignment, and relying on simplistic assumptions such as uniform modality availability across entities. Vision--Language Models (VLMs) offer a powerful way to align diverse modalities within a shared embedding space. We propose Vision--Language Knowledge Graph Embeddings (VL-KGE), a framework that integrates cross-modal alignment from VLMs with structured relational modeling to learn unified multimodal representations of knowledge graphs. Experiments on WN9-IMG and two novel fine art MKGs, WikiArt-MKG-v1 and WikiArt-MKG-v2, demonstrate that VL-KGE consistently improves over traditional unimodal and multimodal KGE methods in link prediction tasks. Our results highlight the value of VLMs for multimodal KGE, enabling more robust and structured reasoning over large-scale heterogeneous knowledge graphs.
论文检索
输入标题、作者或关键词，从 10,462 篇学术成果中精准定位
Time series forecasting is crucial for the World Wide Web and represents a core technical challenge in ensuring the stable and efficient operation of modern web services, such as intelligent transportation and website throughput. However, we have found that existing methods typically employ a strategy of embedding each time step as an independent token. This paradigm introduces a fundamental information bottleneck when processing long sequences, the root cause of which is that independent token embedding destroys a crucial structure within the sequence—what we term as multi-offset temporal correlation. This refers to the fine-grained dependencies embedded within the sequence that span across different time steps, which is especially prevalent in regular Web data. To fundamentally address this issue, we propose a new perspective on time series embedding. We provide an upper bound on the approximate reconstruction performance of token embedding, which guides our design of a concise yet effective Multi-Offset Time Embedding (MOTE) method to mitigate the performance degradation caused by standard token embedding. Furthermore, our MOTE can be integrated into various existing models and serve as a universal building block. Based on this paradigm, we further design a novel forecasting architecture named Time-TK. This architecture first utilizes a Multi-Offset Interactive KAN (MI-KAN) to learn and represent specific temporal patterns among multiple offset sub-sequences. Subsequently, it employs an efficient Multi-Offset Temporal Interaction mechanism (MOTI) to effectively capture the complex dependencies between these sub-sequences, achieving global information integration. Extensive experiments on 14 real-world benchmark datasets, covering domains such as traffic flow and BTC/USDT throughput, demonstrate that Time-TK significantly outperforms all baseline models, achieving state-of-the-art forecasting accuracy.
While Large Language Models (LLMs) excel at many reasoning tasks, their native inability to produce calibrated, multi-class probability distributions limits their use in high-stakes Web applications like content moderation and fraud detection. Existing methods to elicit probabilities from LLMs either sacrifice their crucial Chain-of-Thought (CoT) reasoning capabilities or suffer from poor calibration. To address this, we introduce a new paradigm, Verbalized Probability Distribution, and a novel training framework, RLVP (Reinforcement Learning with Verbalized Probabilities). RLVP fine-tunes an LLM to generate both an interpretable CoT and a complete, verbalized probability distribution. We overcome the ''insufficient reward granularity'' problem in standard Reinforcement Learning (RL) for classification by using soft probabilities from expert tabular models as a dense reward curriculum. Through large-scale joint training on 169 tabular tasks, we demonstrate that a single RLVP-trained model can surpass a strong, task-specific XGBoost baseline on up to 55% of tasks. More importantly, the trained model achieves state-of-the-art few-shot performance on unseen, heterogeneous Web benchmarks that mix structured data with free text, achieving performance comparable to or superior than expert models trained on the same limited data. This showcases a strong capability for generalization and knowledge transfer to complex Web data. Our work presents a viable path toward building general-purpose, probabilistically-sound, and interpretable foundation models for the Web.
Web-scale content is rich in modalities yet frequently incomplete due to device limits, transmission errors, or privacy controls, making learning with missing modalities a core challenge. Prior reconstruction and alignment strategies often fail to preserve a stable class geometry when inputs are partial, leading to anchor drift -- a shift of class prototypes between complete and incomplete views that distorts the shared representation space and degrades generalization. We introduce HiCoD (Hierarchical Consistency-Guided Pro mpt Distillation), which learns a robust, class-anchored semantic space. HiCoD combines: (1) a modality-aware semantic graph that restores cross-modal structure under partial observations; (2) dual-level anchoring that unifies large-language-model–derived global category prototypes with top-K local exemplars to balance cross-modal coherence and modality-specific detail; and (3) multi-level distillation that aligns unimodal features, fused embeddings, and prompt-completed signals within a single anchor space. Across CMU-MOSI, CMU-MOSEI, and additional benchmarks, HiCoD sets a new state of the art under both fixed-pattern and random missingness, improving Acc-2 by up to 6.4 points over MPLMM and remaining robust when key modalities are absent.
The prevalence of sarcasm in multimodal dialogues on the social platforms presents a crucial yet challenging task for understanding the true intent behind online content. Comprehensive sarcasm analysis requires two key aspects: Multimodal Sarcasm Detection (MSD) and Multimodal Sarcasm Explanation (MuSE). Intuitively, the act of detection is the result of the reasoning process that explains the sarcasm. Current research predominantly focuses on addressing either MSD or MuSE as a single task. Even though some recent work has attempted to integrate these tasks, their inherent causal dependency is often overlooked. To bridge this gap, we propose MuVaC, a variational causal inference framework that mimics human cognitive mechanisms for understanding sarcasm, enabling robust multimodal feature learning to jointly optimize MSD and MuSE. Specifically, we first model MSD and MuSE from the perspective of structural causal models, establishing variational causal pathways to define the objectives for joint optimization. Next, we design an alignment-then-fusion approach to integrate multimodal features, providing robust fusion representations for sarcasm detection and explanation generation. Finally, we enhance the reasoning trustworthiness by ensuring consistency between detection results and explanations. Experimental results demonstrate the superiority of MuVaC in public datasets, offering a new perspective for understanding multimodal sarcasm.
Multimodal information extraction (MIE) constitutes a set of essential tasks aimed at extracting structural information from Web texts with integrating images, to facilitate the structural construction of Web-based semantic knowledge. To address the expanding category set including newly emerging entity types or relations on websites, prior research proposed the zero-shot MIE (ZS-MIE) task which aims to extract unseen structural knowledge with textual and visual modalities. However, the ZS-MIE models are limited to recognizing the samples that fall within the unseen category set, and they struggle to deal with real-world scenarios that encompass both seen and unseen categories. The shortcomings of existing methods can be ascribed to two main aspects. On one hand, these methods construct representations of samples and categories within Euclidean space, failing to capture the hierarchical semantic relationships between the two modalities within a sample and their corresponding category prototypes. On the other hand, there is a notable gap in the distribution of semantic similarity between seen and unseen category sets, which impacts the generative capability of the ZS-MIE models. To overcome the above disadvantages, we delve into the generalized zero-shot MIE (GZS-MIE) task and propose the hyperbolic multimodal generative representation learning framework (HMGRL). The variational information bottleneck and autoencoder networks are reconstructed with hyperbolic space for modeling the multi-level hierarchical semantic correlations among samples and prototypes. Furthermore, the proposed model is trained with the unseen samples generated by the decoder, and we introduce the semantic similarity distribution alignment loss to enhance the model's generalization performance. Experimental evaluations on two benchmark datasets underscore the superiority of HMGRL compared to existing baseline methods.
Logs serve as a primary source of information for engineers to diagnose failures in large-scale online service systems. Log parsing, which extracts structured events from massive unstructured log data, is a critical first step for downstream tasks like anomaly detection and failure diagnosis. With advances in large language models (LLMs), leveraging their strong text understanding capabilities has proven effective for accurate log parsing. However, existing LLM-based log parsers all focus on the constant part of logs, ignoring the potential contribution of the variable part to log parsing. This constant-centric strategy brings four key problems. First, inefficient log grouping and sampling with only constant information. Second, a relatively large number of LLM invocations due to constant-based cache, leading to low log parsing accuracy and efficiency. Third, a relatively large number of consumed constant tokens in prompts leads to high LLM invocation costs. At last, these methods only retain placeholders in the results, losing the system visibility brought by variable information in logs. Facing these problems, we propose a variable-centric log parsing strategy named VarParser. Through variable contribution sampling, variable-centric parsing cache, and adaptive variable-aware in-context learning, our approach can efficiently capture the variable parts of logs and leverage their contributions to parsing. By introducing variable units, we preserve rich variable information, enhancing the integrity of log parsing results. Extensive evaluations on large-scale datasets demonstrate that VarParser achieves higher accuracy compared to existing methods, significantly improving parsing efficiency while reducing the LLM invocation costs.
Recommendation systems (RS) aim to retrieve the top-K items most relevant to users, with metrics such as Precision@K and Recall@K commonly used to assess effectiveness. The architecture of an RS model acts as an inductive bias, shaping the patterns the model is inclined to learn. In recent years, numerous recommendation architectures have emerged, spanning traditional matrix factorization, deep neural networks, and graph neural networks. However, their designs are often not explicitly aligned with the top-K objective, thereby limiting their effectiveness. To address this limitation, we propose TopKGAT, a novel recommendation architecture directly derived from a differentiable approximation of top-K metrics. The forward computation of a single TopKGAT layer is intrinsically aligned with the gradient ascent dynamics of the Precision@K metric, enabling the model to naturally improve top-K recommendation accuracy. Structurally, TopKGAT resembles a graph attention network and can be implemented efficiently. Extensive experiments on four benchmark datasets demonstrate that TopKGAT consistently outperforms state-of-the-art baselines. The code is available at https://github.com/StupidThree/TopKGAT.
Graph-based recommender systems have become prevalent in micro-video recommendation by modeling user-item interactions as a bipartite graph. However, these methods face two inherent limitations: (1) their reliance on a fixed, pre-defined graph structure makes them susceptible to noisy interactions, and (2) the multi-modal representations they learn often contain redundant information that is not discriminative enough for the recommendation task. To overcome these issues, we propose a novel Multi-modal Bipartite Graph Structure Learning network (MBGSL), which leverages the information bottleneck principle for robust micro-video recommendation. Specifically, MBGSL first learns adaptive graph structures from multi-modal content (e.g., visual, acoustic, textual) through dedicated graph learners to mitigate noise. Then, it applies an intra-modality information bottleneck to learn minimal sufficient representations within each modality and an inter-modality information bottleneck to capture distinctive information across modalities, thereby eliminating redundancy. Furthermore, the model incorporates collaborative signals through a contrastive learning objective to guide the graph structure learning process. Extensive experiments on three real-world datasets demonstrate that MBGSL achieves state-of-the-art performance, significantly surpassing existing baselines.
Large Language Models (LLMs) have been widely applied across multiple domains for their broad knowledge and strong reasoning capabilities. However, applying them to recommendation systems is challenging since it is hard for LLMs to extract user preferences from large, sparse user-item logs, and real-time per-user ranking over the full catalog is too time-consuming to be practical. Moreover, many existing recommender systems focus solely on ranking items while overlooking explanations, which could help improve predictive accuracy and make recommendations more convincing to users. Inspired by recent works that achieve strong recommendation performance by forecasting near-term item popularity, we propose TRAIL (TRend and explAnation Integrated Learner). TRAIL is a fine-tuned LLM that jointly predicts short-term item popularity and generates faithful natural-language explanations. It employs contrastive learning with positive and negative pairs to align its scores and explanations with structured trend signals, yielding accurate and explainable popularity predictions. Extensive experiments show that TRAIL outperforms strong baselines and produces coherent, well-grounded explanations.
The vulnerability of graph-based recommender systems to spurious correlations has become a significant obstacle to their practical deployment, hindering their robustness in out-of-distribution (OOD) scenarios. While existing approaches offer partial solutions, they are limited by fundamental shortcomings: model-centric approaches reliant on predefined causal graphs often suffer from suboptimal performance due to complex and dynamic environmental influences. These methods typically require identifying an environmental label or performing feature decoupling, but hidden environments are often difficult to model. Furthermore, existing general feature decoupling methods fail to account for the unique structural characteristics of graphs. To overcome these challenges, we advocate for a shift towards explicit, geometrically-grounded disentanglement. Hyperbolic geometry is particularly suited for this task due to its capacity to model the inherent hierarchies of user interests. We introduce C-HyPOD : Causal Hyperbolic Representation Learning with Prototype Orthogonal Disentanglement, a novel framework designed for graph-based OOD recommendation. Unlike traditional methods, C-HyPOD transforms disentanglement into a concrete geometric task. It introduces a global interest space by learning a single set of universal interest prototypes. They provides a superior geometric foundation for ensuring these prototypes are well-separated and semantically distinct. To ensure a complete separation and prevent information leakage, a targeted orthogonality constraint is then applied. This constraint purifies the aggregated causal representation by forcing it to be orthogonal to the spurious representation in the tangent space, thereby eliminating their linear correlation. Extensive experiments on four public datasets demonstrate that C-HyPOD significantly improves OOD robustness and recommendation performance, surpassing state-of-the-art methods.
Collaborative filtering (CF) recommendation has been significantly advanced by integrating Graph Neural Networks (GNNs) and Graph Contrastive Learning (GCL). However, (i) random edge perturbations often distort critical structural signals and degrade semantic consistency across augmented views, and (ii) data sparsity hampers the propagation of collaborative signals, limiting generalization. To tackle these challenges, we propose RaDAR (Relation-\allowbreaka ware Diffusion-\allowbreakAsymmetric Graph Contrastive Learning Framework for Recommendation Systems), a novel framework that combines two complementary view generation mechanisms: a graph generative model to capture global structure and a relation-aware denoising model to refine noisy edges. RaDAR introduces three key innovations: (1) asymmetric contrastive learning with global negative sampling to maintain semantic alignment while suppressing noise; (2) diffusion-guided augmentation, which employs progressive noise injection and denoising for enhanced robustness; and (3) relation-aware edge refinement, dynamically adjusting edge weights based on latent node semantics. Extensive experiments on three public benchmarks demonstrate that RaDAR consistently outperforms state-of-the-art methods, particularly under noisy and sparse conditions. Our code is available at our repository.
Multi-criteria recommender systems (MCRSs) are becoming increasingly important in the Web ecosystem, where platforms such as e-commerce sites and review portals allow users to evaluate items from multiple perspectives. By leveraging criterion-level ratings rather than relying solely on overall scores, MCRSs can enhance personalization and more accurately capture user preferences. However, existing multi-criteria recommendation methods often fail to explicitly model user-specific preferences across criteria or to incorporate item–criterion signals into representation learning. To solve these problems, we propose TaTriGR, a Targeting-Aware Tripartite Graph Recommender, which jointly models user–item–criterion interactions in a unified tripartite graph and integrates user-specific criterion weights through targeting mechanisms. Specifically, TaTriGR encodes user–criterion targeting through personalized weights, while incorporating item–criterion performance as an additional channel to enrich item semantics. A lightweight propagation mechanism then diffuses information across the tripartite structure, and a formal score decomposition shows that predictions satisfy fundamental multi-criteria decision making (MCDM) properties. To further reinforce targeting, TaTriGR introduces two auxiliary objectives: Ideal Point Distillation and Lexicographic Consistency, which encourage criterion-consistent user representations and rankings. Extensive experiments on three real-world datasets demonstrate the effectiveness of TaTriGR, showing relative gains up to 16.1% over the best baseline models. Our implementations are available at https://github.com/nunu1995/TaTriGR.
Watch time prediction (WTP) has emerged as a pivotal task in short video recommendation systems, designed to quantify user engagement through continuous interaction modeling. Predicting users' watch times on videos often encounters fundamental challenges, including wide value ranges and imbalanced data distributions, which can lead to significant estimation bias when directly applying regression techniques. Recent studies have attempted to address these issues by converting the continuous watch time estimation into an ordinal regression task. While these methods demonstrate partial effectiveness, they exhibit notable limitations: (1) The discretization process frequently relies on bucket partitioning, inherently reducing prediction flexibility and accuracy. (2) The interdependencies among different partition intervals remain underutilized, missing opportunities for effective error correction. Inspired by language modeling paradigms, we propose a novel Generative Regression (GR) framework that reformulates WTP as a sequence generation task. Our approach employs structural discretization to enable nearly lossless value reconstruction while maintaining prediction flexibility. Through carefully designed vocabulary construction and label encoding schemes, each watch time is bijectively mapped to a token sequence. To mitigate the training-inference discrepancy caused by teacher-forcing, we introduce a curriculum learning with embedding mixup strategy that gradually transitions from guided to free-generation modes. We test our models extensively on two public datasets, a large-scale offline industrial dataset, and an online A/B test on Kuaishou App with over 400 million daily active users (DAU) and GR consistently outperforms existing state-of-the-art approaches significantly. Our code is available at https://github.com/snailma0229/GR.git.
Personalized Learning Path Planning (PLPP) aims to design adaptive learning paths that align with individual goals. While large language models (LLMs) show potential in personalizing learning experiences, existing approaches often lack mechanisms for goal-aligned planning. We introduce Pxplore, a novel framework for PLPP that integrates a reinforcement-based training paradigm and an LLM-driven educational architecture. We design a structured learner state model and an automated reward function that transforms abstract objectives into computable signals. We train the policy combining supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), and deploy it within a real-world learning platform. Extensive experiments validate Pxplore's effectiveness in producing coherent, personalized, and goal-driven learning paths. We release our code and dataset at https://github.com/Pxplore/pxplore-algo.
Large language models have given rise to interactive recommendation agents (IRAs). Through proactive clarification, tool invocation, and dynamic dialogue, IRAs shift recommender systems from passive prediction to interactive, proactive intelligence. For training IRAs, agentic reinforcement learning offers a natural pathway, as it enables models to learn interactive capabilities directly from environmental feedback without requiring costly annotated data. However, this process faces three key challenges: credit assignment in multi-turn interactions, efficient exploration in large action spaces, and coordinated learning of multiple interactive skills. To tackle this challenge, we present a new preference optimization paradigm, GIMO, which treats the interaction and learning process of IRAs in sparse environments as the continuous fulfillment and stimulation of three intrinsic drives: Autonomy, Competence, and Relatedness. Instead of relying solely on external rewards, GIMO estimates the potential between adjacent states in a generative manner to construct intrinsic rewards. Such a generative motivation structure not only enables fine-grained credit assignment, but also naturally inspires a hint-guided action proposal mechanism that facilitates efficient exploration. Furthermore, during the multi-skill coordination training stage, we introduce an explicit KL regularization into policy orchestration to prevent global policy collapse. Theoretically, we prove that GIMO guarantees global policy consistency, and empirically, experiments in interactive recommendation environments built from three datasets demonstrate that GIMO consistently outperforms existing methods. Our code is anonymously available at https://github.com/XueyangFeng/GIMO.
Multimodal recommendation advocates integrating the multimodal features of items with historical user behaviors to enhance recommendation accuracy across various online media platforms. The majority of existing methods concentrate on leveraging cross-modal learning over multimodal features to augment node representations. However, these approaches are confronted with two key challenges: i) augmented representations offer limited information gain for interactive prediction in the collaborative view, and ii) semantic discrepancy between the collaborative view and modality-augmented features remains inadequately addressed. To overcome these obstacles, we present a new Multi-view Semantic Contrastive Alignment (MSCA) approach for multimodal recommendation, which models and aligns node representations from multiple views. Specifically, we introduce a multi-view semantic pattern encoder that learns basic embeddings from the collaborative view and independently captures augmented semantic patterns from the item-item structural view and intra-modal view. Furthermore, a semantic contrastive alignment task is designed to mitigate the semantic divergence between collaborative embeddings and augmented representations by maximizing the mutual consistency between them, thereby facilitating an effective integration of both. Comprehensive experiments on three benchmark datasets confirm that the proposed MSCA consistently excels over diverse state-of-the-art baselines.
Despite the success of Graph Neural Networks (GNNs) in modeling recommender systems as bipartite graphs, their ability to capture diverse user-item relations remains limited by the sparsity of observed interactions, which fails to reveal the underlying latent intents. We propose IACLR (Intention Alignment via Contrastive Learning for bipartite graph Recommendation), a framework that constructs an Intent-Graph by augmenting the bipartite recommendation graphs with an implicit intent layer. Instead of relying solely on observed edges, IACLR introduces a set of intent nodes that bridge users and items through shared semantic and behavioral patterns. These nodes are used to construct an Intent-Graph, where they act as both intermediaries that enrich structural connectivity and global anchors that summarize latent user interests. Within this graph, IACLR performs contrastive alignment across multiple data views and enforces consistency among users, intents, and items, thereby enhancing robustness under data sparsity. Experiments on benchmark datasets (e.g., Amazon-books and Yelp) demonstrate that IACLR consistently outperforms strong graph-based, revealing its effectiveness in capturing fine-grained user–item relationships and integrating multi-faceted signals. The framework is applicable to various recommendation scenarios, including academic paper recommendations, e-commerce, and content platforms.
Recently, knowledge graphs have been utilised in recommendation systems to improve accuracy by integrating item-side auxiliary information. However, structural user-side knowledge is difficult to construct and integrate due to inherent scarcity and improper granularity. This paper introduces a graph contrastive learning with Semantic transitions-Enhanced DIffusion architecture based on Large Language Models (LLMs) for user-side knowledge-aware Recommendation (SEDIRec). Specifically, our SEDIRec first leverages LLMs to infer user interests from historical behaviors, integrating this user-side information with item-side and collaborative data to construct main views. Then, two contrastive views are generated using diffusion models with semantic transitions: one at the user-side level and the other at the item-side level. For both contrastive views, we integrate user-side or item-side information with collaborative data to generate a user-item graph. Subsequently, each user-item graph is transformed into collaborative data spaces via diffusion models for generating contrastive views. This procedure not only enhances the alignment between user/item-side information and the semantic spaces of collaborative data but also effectively eliminates noise. Extensive experiments on three datasets reveal the superiority of SEDIRec, especially for users with sparse interactions.
Recommendation model performance is intrinsically tied to the quality, volume, and relevance of their training data. To address common challenges like data sparsity and cold start, recent researchs have leveraged data from multiple auxiliary domains to enrich information within the target domain. However, inherent domain gaps can degrade the quality of mixed-domain data, leading to negative transfer and diminished model performance. Existing prevailing model-centric paradigm - which relies on complex, customized architectures - struggles to capture the subtle, non-structural sequence dependencies across domains, leading to poor generalization and high demands on computational resources. To address these shortcomings, we propose Taesar, a data-centric framework for target-aligned sequential regeneration, which employs a contrastive decoding mechanism to adaptively encode cross-domain context into target-domain sequences. It employs contrastive decoding to encode cross-domain context into target sequences, enabling standard models to learn intricate dependencies without complex fusion architectures. Experiments show Taesar outperforms model-centric solutions and generalizes to various sequential models. By generating enriched datasets, Taesar effectively combines the strengths of data- and model-centric paradigms. The code accompanying this paper is available at~ https://github.com/USTC-StarTeam/Taesar.