Advances in social media data dissemination enable the provision of real-time information during a crisis. The information comes from different classes, such as infrastructure damages, persons missing or stranded in the affected zone, etc. Existing methods attempted to classify text and images into various humanitarian categories, but their decision-making process remains largely opaque, which affects their deployment in real-life applications. Recent work has sought to improve transparency by extracting textual rationales from tweets to explain predicted classes. However, such explainable classification methods have mostly focused on text, rather than crisis-related images. In this paper, we propose an interpretable-by-design multimodal classification framework. Our method first learns the joint representation of text and image using a visual language transformer model and extracts text rationales. Next, it extracts the image rationales via the mapping with text rationales. Our approach demonstrates how to learn rationales in one modality from another through cross-modal rationale transfer, which saves annotation effort. Finally, tweets are classified based on extracted rationales. Experiments are conducted over CrisisMMD benchmark dataset, and results show that our proposed method boosts the classification Macro-F1 by 2-35% while extracting accurate text tokens and image patches as rationales. Human evaluation also supports the claim that our proposed method is able to retrieve better image rationale patches (12%) that help to identify humanitarian classes. Our method adapts well to new, unseen datasets in zero-shot mode, achieving an accuracy of 80%.
论文检索
输入标题、作者或关键词，从 13,513 篇学术成果中精准定位
In remote or underserved regions, where road networks are often either unavailable or poorly mapped, and GPS signals are sparse and unreliable, the quality of individual trajectories is severely compromised. In such contexts, web crowdsourced data becomes essential for accurately recovering trajectories and compensating for missing spatial information in the absence of explicit road networks. However, this situation introduces two key challenges:(i) scarcity of data in unseen regions, restricting transfer learning; and (ii) the necessity to infer latent movement structures under roadless conditions. To address these, we propose Region-aware Hierarchical Trajectory Recovery (RHTR) model, designed for location inference from web crowdsourced data in sparse, roadless scenarios. RHTR constructs multi-scale implicit grid-based offset maps from historical location data, with coarse grids capturing global patterns and fine grids refining local details. These sequential representations form region-aware encodings by sampling information around observed points, facilitating trajectory recovery. A coarse-to-fine mechanism leverages contextual information to progressively reconstruct missing segments. Experiments on two public datasets—simulating underserved and disaster-like settings with cross-region transfer—demonstrate that RHTR achieves state-of-the-art performance in trajectory recovery.
The rapid proliferation of emotion-aware web services has necessitated the analysis of multimodal user interactions. However, this introduces new vulnerabilities where adversaries exploit emotional signals to circumvent fraud detection systems. Despite its improved utility, the robustness of multimodal fraud detection against emotion-driven adversarial manipulation remains significantly underexplored. Existing paradigms often treat emotional cues as static features, overlooking the adversary's capability to strategically modulate multimodal signals (e.g., facial micro-expressions, vocal intonation, and textual styles) to mimic genuine behavior. Furthermore, prevalent evaluations are typically confined to unimodal perturbations and fail to account for context-consistent, cross-modal attacks, thereby compromising system reliability in real-world deployments. To bridge this gap, we propose Context-Emotion Adversarial Training (CEAT), a robust framework designed to fortify multimodal fraud detection against emotion-based attacks. CEAT leverages a Transformer-based architecture to synergistically model emotional features (e.g., visual dynamics and acoustic prosody) alongside semantic context derived from text, yielding a unified representation. Crucially, CEAT introduces a context-aware perturbation mechanism that injects noise into the emotional latent space during training. This process preserves semantic consistency while encouraging the learning of emotion-invariant and discriminative representations. Additionally, a contrastive learning objective is integrated to maximize the distributional divergence between genuine and adversarial samples within the latent manifold. Extensive experiments on multimodal benchmarks demonstrate that CEAT significantly outperforms state-of-the-art baselines, exhibiting superior robustness under simulated emotion-driven attack scenarios.
In the rapidly evolving web-based financial ecosystem where digital banking services become critical infrastructure for underserved communities, credit card fraud disproportionately affects vulnerable populations relying on financial platforms. However, previous studies overlook extreme data scarcity conditions, particularly at small-to-medium web banks that serve as crucial gateways for vulnerable communities. This paper addresses the fundamental challenge of building inclusive and secure financial systems operable at true web scale. To overcome this deficiency, we propose a novel web-based fairness-aware federated fraud detection model, CLARF, which utilizes the designed privacy-enhanced representation fusion and fraud-aware contrastive learning modules to enhance detection performance under conditions of data scarcity and label imbalance. Furthermore, current federated fraud detection systems critically neglect vulnerability to backdoor attacks, where malicious actors can implant hidden triggers during model aggregation, compromising system integrity. We propose a novel dynamic web Min-Max adversarial game framework where attackers employ hybrid multi-stage reinforcement learning with multi-dimensional reward mechanisms to dynamically evolve triggers that achieve excellent tradeoff between stealthiness and effectiveness. Defender adapts a closed-loop Selection-Evaluation-Suppression framework where high-reliability clients are selected via Fisher information to carry out reverse trigger engineering. Then clients' confidence scores are calculated as weights to minimize Attack Success Rate (ASR) during aggregation. Extensive experiments on six financial fraud datasets demonstrate the superiority of CLARF model and Min-Max adversarial game paradigm compared with multiple SOTA models.
Accurate semantic segmentation is crucial for rapid risk and severity assessment following flash flood disasters. However, in drone networks deployed for disaster recovery, bandwidth constraints mean the backhaul transmission of massive imagery required for centralized processing is highly prone to causing network congestion. Federated Learning, a decentralized collaborative paradigm that permits drones to process data locally, offers an ideal pathway to address this challenge. Nonetheless, existing FL methods are plagued by the dilemma of excessive communication overhead and insufficient segmentation performance. Therefore, this paper proposes a Communication-Efficient Federated Distillation (CEFD) framework. The core of this framework lies in the design of a lightweight, multi-level knowledge representation. It discards the transmission of bulky parameters, opting instead for the efficient exchange of high-level semantic logits and key intermediate features that have undergone feature-space dimensionality reduction. On the server side, adaptive weighted aggregation is then utilized to construct a robust global knowledge model. Experimental results show that CEFD achieved high segmentation performance on the LSD dataset, attaining an mIoU of 0.6226, a Mean Recall of 0.6532, an Accuracy of 0.9794, and an MAE of 0.0400, while reducing communication overhead by approximately 99.5%. This enables scalable collaborative intelligence for resource-constrained edge devices, a key capability for the future web systems.
Large Language Models (LLMs) perform well across many tasks but degrade when processing large collections of repetitive or highly similar inputs, a common scenario in applications such as near-duplicate search results and large e-commerce catalogs. In these settings, concatenation-based approaches—long-context prompting and supervised fine-tuning—suffer from attention saturation and diminished signal-to-noise ratio, causing models to miss subtle but important distinctions as input size grows. We introduce SEER (Set Encoding for Efficient Representation), a framework that enables LLMs to handle massive sets of near-duplicate items through a single learned token. SEER first encodes individual items with a pretrained embedding model, then aggregates them using an adapter that captures inter-item relationships and preserves fine-grained differences while mitigating redundancy. To ensure both discriminative and generative capabilities, we propose a multi-task alignment strategy that supervises set-level descriptions across multiple semantic dimensions. Experiments on a large-scale e-commerce dataset demonstrate that SEER substantially outperforms in-context and fine-tuned LLM baselines, maintaining stable performance even when processing thousands of highly similar items. These results establish SEER as an effective and scalable approach for LLM processing of dense, redundant input sets.
Fraudulent activities on blockchain networks threaten the integrity and reliability of decentralized finance ecosystems. Accurately identifying malicious nodes such as phishing or ransomware addresses, within large-scale blockchain transaction graphs remains a critical challenge due to their dynamic, sparse, and continuously evolving topologies. Transfer learning offers a powerful paradigm for fraud detection because many fraudulent schemes, including ransomware and phishing, are often orchestrated by overlapping actor groups that share behavioral and structural patterns across networks. Leveraging these shared representations enables knowledge transfer from previously observed fraud types to emerging ones. However, the complex and multi-modal nature of digital financial systems introduces substantial challenges for graph-based transfer learning. Fraudulent activities are shaped by diverse modalities including graph structure, transaction sequences, temporal price dynamics, and textual metadata, while distributional shifts frequently occur across time and platforms. Existing graph transfer learning methods struggle to model such multi-modal dependencies and to align divergent feature distributions. To tackle these challenges, we develop a Multi-mOdal Enhanced Graph Transfer Learning (MOE-GTL) framework which incorporates graph, temporal, and textual modalities for fraudulent node detection. We further introduce Temporal-aware Maximum Mean Discrepancy (TMMD), a regularization mechanism that explicitly aligns multi-modal feature distributions between source and target graphs over time. Extensive experiments reveal that our MOE-GTL model notably improves the accuracy of fraudulent node classifications on Ethereum and Solana transaction graphs.
Retrieval-augmented generation (RAG) is an effective approach to enhancing the factual accuracy of radiology reports. However, existing methods primarily model coarse-grained image–report correspondences, ignoring semantic relations among reports that capture hierarchical and fine-grained pathological knowledge. As a result, the learned representations fail to reflect detailed clinical semantics, causing factual inconsistencies in generated reports. Therefore, we propose a multi-granularity knowledge-integrated RAG framework for radiology reports. Specifically, we utilize multi-granularity semantic similarities, derived from the text modality, to adjust the original cross-modal contrastive learning loss. This guides the multimodal retriever to learn a finer-grained clinical semantic alignment. Then, we utilize cross attention to obtain enhanced visual features by integrating the retrieved reports with the original images, thus enhancing the factual accuracy of report generation. The effectiveness of our method was verified on two widely used benchmarks, achieving superior performance in both language generation and key clinical metrics.
Accurate estimation of delivery time (EDT) is a critical factor in web e-commerce user experience. The pursuit of higher EDT accuracy has predominantly centered on designing increasingly complex model architectures. While valuable, this architecture-centric paradigm creates a tension between its high iteration costs and the industrial demand for agile deployment. This work, therefore, explores a complementary dimension: enhancing model performance by optimizing the learning process itself. We propose EDTF, a novel, plug-and-play composite learning framework that empowers existing models by augmenting their learning objective. EDTF first transforms the traditional regression problem into a structured ordinal classification task to address the training difficulties inherent in direct regression and preserve temporal order. It then introduces a cross-view consistency paradigm, decomposing the prediction task into two related views: the macroscopic end-to-end delivery time and the microscopic next-hop duration. By enforcing a self-supervised signal that aligns the sum of future next-hop durations with the overall EDT, our framework enables models to learn more robust temporal representations without extra features. Extensive experiments on a large-scale industrial dataset show that EDTF, as a plugin, consistently enhances performance and accelerates convergence across five diverse architectures. Critically, an EDTF-optimized model has been successfully deployed in a live production environment, demonstrating significant improvements over its predecessor. This work thus presents a validated and valuable new paradigm for the economical and efficient application of web services reliant on trajectory-based forecasting, from e-commerce to ride-hailing and food delivery.
Behavioral patterns captured in embeddings learned from interaction data are pivotal across various stages of production recommender systems. However, in the initial retrieval stage, practitioners face an inherent tradeoff between embedding expressiveness and the scalability and latency of serving components, resulting in the need for representations that are both compact and expressive. To address this challenge, we propose a training strategy for learning high-dimensional sparse embedding layers in place of conventional dense ones, balancing efficiency, representational expressiveness, and interpretability. To demonstrate our approach, we modified the production-grade collaborative filtering autoencoder ELSA, achieving up to 10× reduction in embedding size with no loss of recommendation accuracy, and up to 100× reduction with only a 2.5% loss. Moreover, the active embedding dimensions reveal an interpretable inverted-index structure that segments items in a way directly aligned with the model's latent space, thereby enabling integration of segment-level recommendation functionality (e.g., 2D homepage layouts) within the candidate retrieval model itself. Source codes, additional results, as well as a live demo are available at https://github.com/zombak79/compressed\_elsa.
With the rise of modern search and recommendation platforms, insufficient collaborative information of cold-start items exacerbates the Matthew effect of existing platform items, challenging platform diversity and becoming a longstanding issue. Existing methods align items' side content with collaborative information to transfer collaborative signals from high-popularity items to cold-start items. However, these methods fail to account for the asymmetry between collaboration and content, nor the fine-grained differences among items. To address these issues, we propose COINS, an item representation enhancement approach based on fused alignment of semantic IDs. Specifically, we use RQ-OPQ encoding to quantize item content and collaborative information, followed by a two-step alignment: RQ encoding transfers shared collaborative signals across items, while OPQ encoding learns items' differentiated information. Comprehensive offline experiments on large-scale industrial datasets demonstrate COINS's superiority, and rigorous online A/B tests confirm statistically significant improvements.
Large-scale digital platforms generate billions of timestamped user-item interactions (events) that are crucial for predicting user attributes in, e.g., fraud prevention and recommendations. While self-supervised learning (SSL) effectively models the temporal order of events, it typically overlooks the global structure of the user-item interaction graph. To bridge this gap, we propose three model-agnostic strategies for integrating this structural information into contrastive SSL: enriching event embeddings, aligning client representations with graph embeddings, and adding a structural pretext task. Experiments on four financial and e-commerce datasets demonstrate that our approach consistently improves the accuracy (up to a 2.3% AUC) and reveals that graph density is a key factor in selecting the optimal integration strategy.
Audio deepfakes generated by modern TTS and voice conversion systems are increasingly difficult to distinguish from real speech, raising serious risks for security and online trust. While state-of-the- art self-supervised models provide rich multi-layer representations, existing detectors treat layers independently and overlook temporal and hierarchical dependencies critical for identifying synthetic arte- facts. We propose HierCon, a hierarchical layer attention framework combined with margin-based contrastive learning that models de- pendencies across temporal frames, neighbouring layers, and layer groups, while encouraging domain-invariant embeddings. Evalu- ated on ASVspoof 2021 DF and In-the-Wild datasets, our method achieves state-of-the-art performance (1.93% and 6.87% EER), im- proving over independent layer weighting by 36.6% and 22.5% re- spectively. The results and attention visualisations confirm that hierarchical modelling enhances generalisation to cross-domain generation techniques and recording conditions.
Generative models are increasingly being explored in click-through rate (CTR) prediction field to overcome the limitations of the conventional discriminative paradigm, which rely on a simple binary classification objective. However, existing generative models typically confine the generative paradigm to the training phase, primarily for representation learning. During online inference, they revert to a standard discriminative paradigm, failing to leverage their powerful generative capabilities to further improve prediction accuracy. This fundamental asymmetry between the training and inference phases prevents the generative paradigm from realizing its full potential. To address this limitation, we propose the Symmetric Masked Generative Paradigm for CTR prediction (SGCTR), a novel framework that establishes symmetry between the training and inference phases. Specifically, after acquiring generative capabilities by learning feature dependencies during training, SGCTR applies the generative capabilities during online inference to iteratively redefine the features of input samples, which mitigates the impact of noisy features and enhances prediction accuracy. Extensive experiments validate the superiority of SGCTR, demonstrating that applying the generative paradigm symmetrically across both training and inference significantly unlocks its power in CTR prediction.
The proliferation of ride-hailing services has fundamentally transformed urban mobility patterns, making accurate ride-hailing forecasting crucial for optimizing passenger experience and urban transportation efficiency. However, ride-hailing forecasting faces significant challenges due to geospatial heterogeneity and high susceptibility to external events. This paper proposes MVGR-Net (Multi-View Geospatial Representation Learning), a novel framework that addresses these challenges through a two-stage approach. In the pre-training stage, we learn comprehensive geospatial representations by integrating Points-of-Interest and temporal mobility patterns to capture regional characteristics from both semantic attribute and temporal mobility pattern views. The forecasting stage leverages these representations through a prompt-empowered framework that fine-tunes Large Language Models while incorporating external events. Extensive experiments on DiDi's real-world datasets demonstrate the state-of-the-art performance.
In modern e-commerce search systems, dense retrieval has become an indispensable component. By computing similarities between query and item (product) embeddings, it efficiently selects candidate products from large-scale repositories. With the breakthroughs in large language models (LLMs), mainstream embedding models have gradually shifted from BERT to LLMs for more accurate text modeling. However, these models still adopt direct-embedding methods, and the semantic accuracy of embeddings remains inadequate. Therefore, contrastive learning is heavily employed to achieve tight semantic alignment between positive pairs. Consequently, such models tend to capture statistical co-occurrence patterns in the training data, biasing them toward shallow lexical and semantic matches. For difficult queries exhibiting notable lexical disparity from target items, the performance degrades significantly. In this work, we propose the Large Reasoning Embedding Model (LREM), which novelly integrates reasoning processes into representation learning. For difficult queries, LREM first conducts reasoning to achieve a deep understanding of the original query, and then produces a reasoning-augmented query embedding for retrieval. This reasoning process effectively bridges the semantic gap between original queries and target items, significantly improving retrieval accuracy. Specifically, we adopt a two-stage training process: the first stage optimizes the LLM on carefully curated Query-CoT-Item triplets with SFT and InfoNCE losses to establish preliminary reasoning and embedding capabilities, and the second stage further refines the reasoning trajectories via reinforcement learning (RL). Extensive offline and online experiments validate the effectiveness of LREM, leading to its deployment on China's largest e-commerce platform since August 2025.
Generative Recommenders (GRs), exemplified by the Hierarchical Sequential Transduction Unit (HSTU), have emerged as a powerful paradigm for modeling long user interaction sequences. However, we observe that their ''flat-sequence'' assumption overlooks the rich, intrinsic structure of user behavior. This leads to two key limitations: a failure to capture the temporal hierarchy of session-based engagement, and computational inefficiency, as dense attention introduces significant noise that obscures true preference signals within semantically sparse histories, which deteriorates the quality of the learned representations. To this end, we propose a novel framework named HPGR (Hierarchical and Preference-aware Generative Recommender), built upon a two-stage paradigm that injects these crucial structural priors into the model to handle the drawback. Specifically, HPGR comprises two synergistic stages. First, a structure-aware pre-training stage employs a session-based Masked Item Modeling (MIM) objective to learn a hierarchically-informed and semantically rich item representation space. Second, a preference-aware fine-tuning stage leverages these powerful representations to implement a Preference-Guided Sparse Attention mechanism, which dynamically constrains computation to only the most relevant historical items, enhancing both efficiency and signal-to-noise ratio. Empirical experiments on a large-scale proprietary industrial dataset from APPGallery and an online A/B test verify that HPGR achieves state-of-the-art performance over multiple strong baselines, including HSTU and MTGR.
Recommender Systems (RS) are fundamental to modern online services. While most existing approaches optimize for short-term engagement, recent work has begun to explore reinforcement learning (RL) to model long-term user value. However, these efforts face significant challenges due to the vast, dynamic action spaces inherent in RS, which hinder stable policy learning. To resolve this bottleneck, we introduce Hierarchical Semantic RL (HSRL), which reframes RL-based recommendation over a fixed Semantic Action Space (SAS). HSRL encodes items as Semantic IDs (SIDs) for policy learning, and maps SIDs back to their original items via a fixed lookup during execution. To align decision-making with SID generation, the Hierarchical Policy Network (HPN) operates in a coarse-to-fine manner, employing hierarchical residual state modeling to refine each level's context from the previous level's residual, thereby reducing representation–decision mismatch. In parallel, a Multi-level Critic (MLC) provides token-level value estimates, enabling fine-grained credit assignment. Across public benchmarks and a large-scale production dataset from a leading short-video advertising platform, HSRL consistently surpasses state-of-the-art baselines. In online deployment over a 7-day A/B testing, it delivers an 18.421% ADVV lift and a 1.251% increase in Revenue, supporting HSRL as a scalable paradigm for RL-based recommendation.
In real-world industrial scenarios, large language models (LLMs) require Continuous Learning (CL) to adapt to diverse tasks as opera- tional requirements diversify, demanding self-evolution capabilities to autonomously refine their knowledge and adapt to dynamic envi- ronments. However, existing CL approaches, such as replay-based and parameter isolation techniques, struggle with the catastrophic forgetting problem: new task training degrades performance on prior tasks due to the model's adaptation to new data distributions, which weakens its generalization to old tasks. To address this issue, we propose a novel parameter-efficient adversarial MoE framework, MoE-CL, for industrial-scale self-evolving continual instruction tuning of LLMs. Specifically, MoE-CL employs a dual-expert archi- tecture to enable self-evolution: a dedicated LoRA expert for each task to preserve task-specific knowledge, ensuring parameter inde- pendence and mitigating forgetting, and a shared LoRA expert to facilitate cross-task knowledge transfer. Specifically, a task-aware discriminator within a Generative Adversarial Network (GAN) is integrated into the shared expert to suppress task-irrelevant noise, ensuring only task-aligned knowledge is transferred during se- quential task training. Through adversarial training, the shared ex- pert learns generalized representations that mimic the task-aware discriminator, while dedicated experts retain task-specific details, balancing knowledge retention and cross-task generalization—key to the model's self-evolution by autonomously optimizing knowl- edge integration across tasks. Extensive experiments on a public MTL5 benchmark and an industrial Tencent3 benchmark validate MoE-CL's effectiveness in self-evolving continual learning. In real- world A/B testing on content compliance review in the Tencent Video Platform, MoE-CL reduced manual review costs by 15.3%,demonstrating its applicability for large-scale industrial deploy- ment where self-evolution is critical for adapting to evolving op- erational demands. Implementation code is publicly available at https://github.com/BAI-LAB/MoE-CL.
In the rapidly evolving landscape of e-commerce, visual search has become a cornerstone of user experience, enabling customers to find products using images rather than traditional text queries. However, a comprehensive analysis reveals a persistent challenge: nearly half of retrieval failures stem from systems that prioritize superficial visual similarity over semantic relevance, resulting in frustrating user experiences where searches return visually similar but functionally different products. This limitation becomes particularly acute in Business-to-Business environments, where incorrect product recommendations can have significant operational and safety implications. In this paper, we propose a novel solution, Adaptive Retrieval with Category-aware Hierarchical sEmantic Refinement (ARCHER), which presents a novel multimodal retrieval framework that addresses these challenges through progressive semantic alignment. Unlike existing approaches that treat all visual similarities equally, ARCHER employs a sophisticated three-stage learning strategy that systematically builds from coarse-grained category understanding to fine-grained product discrimination. The framework begins with Proto-Align Enhancement to establish foundational visual-textual correspondences, progresses through Cross-View Learning to develop robust viewpoint-invariant representations, and culminates with Margin-based Representation Enhancement that learns to distinguish between visually similar but functionally distinct products. Most significantly, the framework has been successfully deployed on Alibaba.com's B2B platform since December 2024, where it serves millions of daily queries and has achieved a measurable 2.1% improvement in click-through rates.