We present a role-aware virtual agent navigational interaction that generates consistent, role-aligned movement behaviors. Our approach leverages Multimodal Large Language Models (MLLMs) to interpret multimodal inputs including scene information, user state, and high-level language role instruction, producing discrete navigation decisions and stylized planning path. Our approach enables virtual agents to behave consistently with narrative roles and respond to dynamic actions, such as playing a hide-and-seek taking into account the agent's role and the user's possible intention. Our approach demonstrates how MLLMs can go beyond language-based interaction to support embodied, spatial, and role-aware agent behaviors in immersive environments such as augmented reality.
论文检索
输入标题、作者或关键词，从 7,537 篇学术成果中精准定位
Supporting a wide variety of motion styles is critical for creating diverse virtual characters, but current methods either require large stylized datasets or pre-trained models that cannot generalize beyond their training distribution. We present STyMo, a few-shot approach that learns motion style from only seconds of paired data and trains in one to two minutes. Our key insight is to decompose style into two components: a static component capturing time-invariant posture, and a temporal component capturing frame-wise dynamics. This decomposition yields an interpretable system where posture intensity, temporal exaggeration, and per-body-region style can be adjusted at runtime. Furthermore, the reduction in required training data and computation time structurally permits an iterative authoring workflow. To ensure robustness on arbitrary inputs, we further introduce a stylizability gate that automatically prevents artifacts on out-of-distribution motions. We demonstrate results across diverse motion styles, from subtle emotional variations to exaggerated character archetypes, and release our processed paired dataset to facilitate future research. The source code used in this paper can be found at: https://github.com/facebookresearch/STyMo
We present ST-FLIP, a spatiotemporal extension of the Fluid-Implicit Particle (FLIP) method for incompressible free-surface and two-phase liquid simulation. ST-FLIP enables time steps up to an order of magnitude larger than those typically used in CFL-constrained solvers, while preserving detailed flow structures and visual fidelity. It addresses a common failure mode of large time steps in hybrid particle-grid liquid solvers: temporal under-sampling of particle motion produces aliasing-driven free-surface artifacts after projection. Our key idea is to interpret particles as samples in four-dimensional space-time: in addition to standard spatial jittering, we randomize particle positions along the time axis as well and perform particle-to-grid deposition using a separable 4D kernel. This yields a Monte Carlo estimator of perstep time-slab-integrated particle quantities. Although particles are treated as samples in 4D space-time, our approach works as a lightweight plugin by collapsing to slab-integrated 3D grid fields for projection. Building on recent particle-based phase-field work, we reuse the particle-to-grid weight accumulators as a conceptual space-time phase field, providing variable-coefficient projection weights and eliminating the need for per-step surface reconstruction. The method can be easily integrated into existing FLIP/PIC or APIC solvers with negligible additional computational cost per time step. The effectiveness of our approach is demonstrated through a series of comparisons with state-of-the-art solvers, yielding several-fold speedups for multi-billion-particle simulations at high effective 3D resolutions on a single workstation.
Selection is a core operation in interactive image editing, enabling tasks such as composition or manipulation. To be practically useful, a user should be able to specify and disambiguate the desired selection region through either text- or click-based interactions, and the system should support selecting not only objects but also other criteria, such as materials. Material-based selection can be particularly valuable for tasks like re-texturing surfaces or consistently editing all instances of a specific material in a scene. However, existing vision–language-model (VLM) based selection methods are largely object-centric and typically support only a single interaction modality, limiting their applicability in real editing workflows. In this work, we thus present Mask Any Object And Material (MAOAM), a unified selection framework that enables precise object- and material-level selection across both text- and click-based interactions. MAOAM leverages a VLM with a segmentation head to produce pixel-accurate masks from user prompts: the VLM interprets the user’s selection intent — object- or material-level — and encodes visual entities, attributes, and spatial relations, while the segmentation head decodes the VLM’s output token into a mask. A key challenge is that material selection datasets with text annotations are unavailable. We therefore propose a scalable data generation pipeline: we collect real and synthetic images with material masks, then leverage VLMs to generate material descriptions with rich visual-semantic information. Using the generated data, we train MAOAM with a multi-task objective over click- and text-based selection, along with an auxiliary VQA task derived from the material descriptions to facilitate deeper material understanding. Despite being trained with uni-modal prompts, our model exhibits an emergent improvement in selection quality when combining text and clicks at inference time, enabling more flexible image editing workflows. Experiments demonstrate accurate and coherent selections across diverse objects, materials, and interaction scenarios, highlighting robustness in practice.
Text-to-image diffusion models generate images by gradually converting white Gaussian noise into a natural image. White Gaussian noise is well suited for producing diverse outputs from a single text prompt due to its absence of structure. However, this very property limits control over, and predictability of, specific visual attributes, as the noise is not human-interpretable. In this work, we investigate the characteristics of the input noise in diffusion models. We show that, although all frequencies in white Gaussian noise have comparable statistical energy, low-frequency components primarily determine the image’s global structure and color composition, while high-frequency components control finer details. Building on this observation, we demonstrate that simple manipulations of the low-frequency noise using low-frequency image priors can effectively condition the generation process to reconstruct these low-frequency visual cues. This allows us to define a simple, training-free method with minimal overhead that steers overall image structure and color, while letting high-frequency components freely emerge as fine details, enabling variability across generated outputs.
Advances in large language models (LLMs) have sparked interest in automating parametric CAD modeling through natural language. Existing LLM-based approaches often treat CAD modeling as flat text generation, overlooking the hierarchical structure and geometric constraints inherent in CAD programs. We present CAD-Factory, a Text-to-CAD generation system for language-driven CAD modeling that explicitly models the structural and parametric semantics of CAD programs. Our core contribution is a new formulation of CAD generation as structured program synthesis, coupled with a learnable hierarchical CAD program representation that disentangles structural topology from parameter instantiation. Building on this formulation, CAD-Factory adopts a manager–programmer–reviewer architecture: a planning agent infers program hierarchy, a coding agent instantiates symbolic and numerical parameters, and an evaluation module enforces structural validity and geometric feasibility, which supports structure-aware reasoning, constraint-consistent generation, and interpretable program synthesis. An editor–viewer loop further enables part-aware code refinement through visual feedback, supporting iterative and controllable design workflows. We also contribute a process-oriented annotation pipeline and a Text–CAD dataset with parameter-free or partially specified descriptions that reflect real-world design expression. Extensive experiments demonstrate that CAD-Factory significantly improves structural correctness and geometric consistency across diverse CAD code generation and editing tasks, establishing a structured, interpretable, and robust framework for AI-assisted design.
Computer-Aided Design (CAD) models, given their compactness and precision, remain the industry standard for designing and fabricating engineering objects. However, language-guided CAD editing is still in its infancy, largely due to missing semantic connection between user commands and underlying shape geometry, a problem exacerbated by the shortage of paired text-and-edit CAD datasets. While recent Multimodal Large Language Models (mLLMs) have attempted to bridge this gap, their reliance on CAD construction history –often an expensive and hard to obtain input– severely limits their expressiveness and restricts their usage. We present B-repLer, a novel framework that directly connects natural language with editing CAD models by operating in a learned latent space. Importantly, our approach bypasses the need for construction history, enabling semantic edits on a wide range of geometries, from simple prismatic parts to complex freeform shapes defined by B-Spline surfaces. To facilitate this research, we introduce BrepEDIT-240K, the first large-scale dataset for this task. We demonstrate how this paired dataset can be automatically generated, (user) validated, and scaled by leveraging existing CAD tools, in conjunction with mLLMs, to create the required paired data without relying on any external annotations. Our results demonstrate that B-repLer can accurately perform complex edits on complex CAD shapes, even when the input edit specifications are high-level and ambiguous to interpret, consistently producing valid, high-quality CAD outputs enabling a class of text-guided edits not previously possible. Project page is at https://yilinliu77.github.io/brepler.github.io/.
Cultural context profoundly shapes how people interpret online content, yet vision–language models (VLMs) remain predominantly trained through Western or English-centric lenses. This limits their fairness and cross-cultural robustness in tasks like hateful meme detection. We introduce a systematic evaluation framework designed to diagnose and quantify the cross-cultural robustness of state-of-the-art VLMs across multilingual meme datasets, analyzing three axes: (i) learning strategy (zero-shot vs. one-shot), (ii) prompting language (native vs. English), and (iii) translation effects on meaning and detection. Results show that the common ''translate-then-detect'' approach deteriorate performance, while culturally aligned interventions — native-language prompting and one-shot learning — significantly enhance detection. Our findings reveal systematic convergence toward Western safety norms and provide actionable strategies to mitigate such bias, guiding the design of globally robust multimodal moderation systems.
Popularity bias is a pervasive challenge in recommender systems, where a few popular items dominate attention while the majority of less popular items remain underexposed. This imbalance can reduce recommendation quality and lead to unfair item exposure. Although existing mitigation methods address this issue to some extent, they often lack transparency in how they operate. In this paper, we propose a post-hoc approach, PopSteer, that leverages a Sparse Autoencoder (SAE) to both interpret and mitigate popularity bias in recommendation models. The SAE is trained to replicate a trained model's behavior while enabling neuron-level interpretability. By introducing synthetic users with strong preferences for either popular or unpopular items, we identify neurons encoding popularity signals through their activation patterns. We then steer recommendations by adjusting the activations of the most biased neurons. Experiments on three public datasets with a sequential recommendation model demonstrate that PopSteer significantly enhances fairness with minimal impact on accuracy, while providing interpretable insights and fine-grained control over the fairness–accuracy trade-off.
Large Language Models (LLMs) are now widely deployed across modern web services, but their safe and trustworthy use in real-world settings critically depends on accurate alignment with human preferences. Preference alignment is typically achieved using methods such as reinforcement learning or direct preference optimization (DPO), whose effectiveness in practice hinges on the quality of labeled preference data. However, a fundamental practical challenge remains: preference datasets inevitably contain noise. Through a systematic analysis of mainstream preference datasets, we find that roughly 25% of preference pairs show clear inconsistencies between reward-model evaluations and human annotations. Such inconsistent examples do not convey reliable preference signals; training directly on them therefore not only fails to improve alignment but can even degrade model behavior. To address this problem, we propose Noise-Aware Preference Alignment for LLMs via Confidence and Polarity Reweighting (AlignCP), a fully automated, human-free framework for noise-aware preference alignment. AlignCP derives two interpretable metrics from reward-model outputs: Confidence, which measures the reliability of each preference judgment, and Polarity, which evaluates whether the reward-model ranking agrees with the original human label. These metrics are combined to assign a training weight to each sample—amplifying high-confidence, label-consistent pairs while down-weighting or discarding low-confidence, contradictory, or otherwise noisy pairs. Unlike methods that rely on human re-inspection, repeated relabeling, or heavy computational reruns, AlignCP performs automated data-quality control with minimal overhead and no human intervention. Empirical results show that AlignCP substantially outperforms existing data-centric alignment approaches on standard preference benchmarks and remains more robust under noisy supervision.
Misinformation on the web increasingly appears in multimodal forms, combining text, images, and OCR-rendered content in ways that amplify harm to public trust and vulnerable communities. While prior fact-checking systems often rely on unimodal signals or shallow fusion strategies, modern misinformation campaigns operate across modalities and require models that can reason over subtle cross-modal inconsistencies in a transparent and responsible manner. We introduce MultiCheck, a lightweight and interpretable framework for multimodal fact verification that jointly analyzes textual, visual, and OCR evidence. At its core, MultiCheck employs a relational fusion module based on element-wise difference and product operations, allowing for explicit cross-modal interaction modeling with minimal computational overhead. A contrastive alignment objective further helps the model distinguish between supporting and refuting evidence while maintaining a small memory and energy footprint, making it suitable for low-resource deployment. Evaluated on the Factify-2 (5-class) and MOCHEG (3-class) benchmarks, MultiCheck achieves substantial performance improvement and remains robust under noisy OCR and missing modality conditions. Overall, MultiCheck is efficient, interpretable, and robust for multimodal verification. Our code is available at the following: https://github.com/Adityakishore09/MultiCheck_WWW-2026 GitHub repository.
Advances in social media data dissemination enable the provision of real-time information during a crisis. The information comes from different classes, such as infrastructure damages, persons missing or stranded in the affected zone, etc. Existing methods attempted to classify text and images into various humanitarian categories, but their decision-making process remains largely opaque, which affects their deployment in real-life applications. Recent work has sought to improve transparency by extracting textual rationales from tweets to explain predicted classes. However, such explainable classification methods have mostly focused on text, rather than crisis-related images. In this paper, we propose an interpretable-by-design multimodal classification framework. Our method first learns the joint representation of text and image using a visual language transformer model and extracts text rationales. Next, it extracts the image rationales via the mapping with text rationales. Our approach demonstrates how to learn rationales in one modality from another through cross-modal rationale transfer, which saves annotation effort. Finally, tweets are classified based on extracted rationales. Experiments are conducted over CrisisMMD benchmark dataset, and results show that our proposed method boosts the classification Macro-F1 by 2-35% while extracting accurate text tokens and image patches as rationales. Human evaluation also supports the claim that our proposed method is able to retrieve better image rationale patches (12%) that help to identify humanitarian classes. Our method adapts well to new, unseen datasets in zero-shot mode, achieving an accuracy of 80%.
Child mental health screening faces growing challenges from rising psychological problems and limited professional access. Traditional self-report tools rely on verbal ability and self-awareness, limiting their validity in younger children. Projective drawing tests offer a nonverbal alternative, with the Draw-A-Person (DAP) test widely used to elicit psychological cues from drawings. While translating DAP test into Web-based screening, multimodal large language models (MLLMs) are emerging as a key enabling mechanism. However, their ability to deliver construct-level clarity and interpretive consistency for responsible deployment has not been systematically evaluated. To address this gap, we propose DAPWeb, a construct-aligned evaluation framework that assesses whether MLLMs can reliably support DAP-based child mental screening in Web environments. DAPWeb introduces a clinically grounded benchmark derived from real drawings and defines task-structured evaluation across six psychological constructs and three essential screening abilities: determination, detection, and comparison. The metrics emphasize construct validity and cross-drawing consistency, reflecting real-world early-screening workflows. Experiments across MLLMs reveal substantial gaps from human experts in most abilities. Yet, the comparable performance on determination suggests that DAP test can serve as a feasible component of Web-based early screening under structured interpretation. Thus, DAPWeb provides a replicable paradigm for responsible Web AI in child mental health.
Web-based platforms are becoming a primary channel for psychological support, yet most LLM-driven chatbots remain opaque, single-stage, and weakly grounded in established therapeutic practice. To address this gap, we present XInsight, a multi-agent framework that models psychological support as a stage-consistent workflow aligned with the classical Exploration-Insight-Action paradigm. Building on structured client representations, XInsight orchestrates specialized agents under a unified Reason-Intervene-Reflect cycle: an Exploration agent organizes background and concerns into a structured Case Conceptualization Form, a Routing agent performs Adaptive Therapeutic Routing (ATR) across SFBT, CBT, and MBCT, a unified Therapeutic agent executes school-consistent submodules, and a Consolidation agent guides review, skill integration, and relapse-prevention planning. A Recording agent continuously transforms open-ended web dialogues into standardized psychological artifacts, enhancing interpretability, continuity, and accountability. To support transparent assessment, we introduce XInsight-Bench with a Scale-Guided LLM Evaluation (SGLE) protocol that combines therapy-specific clinical scales with general counseling criteria. Experiments show improved paradigm alignment, multi-therapy integration, interaction depth, and interpretability over existing multi-agent counseling systems, indicating that XInsight provides a practical blueprint for integrating counseling-inspired support agents into web applications for digital well-being.
Large Language Models (LLMs) are increasingly used for toxicity assessment in online moderation systems, where fairness across demographic groups is essential for equitable treatment. However, LLMs often produce inconsistent toxicity judgements for subtle expressions, particularly those involving implicit hate speech, revealing underlying biases that are difficult to correct through standard training. This raises a key question that existing approaches often overlook: when should corrective mechanisms be invoked to ensure fair and reliable assessments? To address this, we propose FairToT, an inference-time framework that enhances LLM fairness through prompt-guided toxicity assessment. FairToT identifies cases where demographic-related variation is likely to occur and determines when additional assessment should be applied. In addition, we introduce two interpretable fairness indicators that detect such cases and improve inference consistency without modifying model parameters. Experiments on benchmark datasets show that FairToT reduces group-level disparities while maintaining stable and reliable toxicity predictions, demonstrating that inference-time refinement offers an effective and practical approach for fairness improvement in LLM-based toxicity assessment systems. The source code can be found at https://aisuko.github.io/fair-tot/.
As web platforms evolve towards greater personalization and emotional complexity, conversational agents must transcend superficial empathy to demonstrate identity-aware emotional reasoning. However, existing systems face two limitations: (1) reliance on situation-centric datasets lacking persistent user identity, which hampers the capture of personalized affective nuances; and (2) dependence on opaque, coarse reward signals that hinder development of verifiable empathetic reasoning. To address these gaps, we introduce KardiaBench, a large-scale user-grounded benchmark comprising 178,080 QA pairs across 22,080 multi-turn conversations anchored to 671 real-world profiles. The dataset is constructed via a model-in-the-loop pipeline with iterative rubric-guided refinement to ensure psychological plausibility and persona consistency. This progressive empathy pipeline that integrates user comprehension, contextual reasoning, and emotion perception into conversations, followed by iterative critique and rubric-based refinement to ensure psychological plausibility, emotional fidelity, and persona consistency. Building on this, we propose Kardia-R1, a framework that trains models for interpretable, stepwise empathetic cognition. Kardia-R1 leverages Rubric-as-Judge Empathetic Reinforcement Learning (Rubric-ERL), a GRPO-based method that uses explainable, human-aligned rubric rewards to tightly couple user understanding, emotional inference, and supportive response generation. Extensive experiments across four LLM backbones demonstrate that Kardia-R1 consistently outperforms other methods in emotion accuracy, empathy, relevance, persona consistency, and safety.
Graphs provide a powerful basis for modeling Web-based relational data, with expressive GNNs to support the effective learning in dynamic web environments. However, real-world deployment is hindered by pervasive out-of-distribution (OOD) shifts, where evolving user activity and changing content semantics alter feature distributions and labeling criteria. These shifts often lead to unstable or overconfident predictions, undermining the trustworthiness required for Web4Good applications. Achieving reliable OOD generalization demands principled and interpretable uncertainty estimation; however, existing methods are largely post-hoc, insensitive to distribution shifts, and unable to explain where uncertainty arises especially in high-stakes settings. To address these limitations, we introduce SpIking GrapH predicTive coding (SIGHT ), an uncertainty-aware plug-in graph learning module for reliable OOD Generalization. SIGHT performs iterative, error-driven correction over spiking graph states, enabling models to expose internal mismatch signals that reveal where predictions become unreliable. Across multiple graph benchmarks and diverse OOD scenarios, SIGHT consistently enhances predictive accuracy, uncertainty estimation, and interpretability when integrated with GNNs.
The rapid growth of video content on platforms such as TikTok and YouTube has intensified the spread of multimodal hate speech, where harmful cues emerge subtly and asynchronously across visual, acoustic, and textual streams. Existing research primarily focuses on video-level classification, leaving the practically crucial task of temporal localisation, identifying when hateful segments occur, largely unaddressed. This challenge is even more noticeable under weak supervision, where only video-level labels are available, and static fusion or classification-based architectures struggle to capture cross-modal and temporal dynamics. To address these challenges, we propose MultiHateLoc, the first framework designed for weakly-supervised multimodal hate localisation. MultiHateLoc incorporates (1) modality-aware temporal encoders to model heterogeneous sequential patterns, including a tailored text-based preprocessing module for feature enhancement; (2) dynamic cross-modal fusion to adaptively emphasise the most informative modality at each moment and a cross- modal contrastive alignment strategy to enhance multimodal feature consistency; (3) a modality-aware MIL objective to identify discriminative segments under video-level supervision. Despite relying solely on coarse labels, MultiHateLoc produces fine-grained, interpretable frame-level predictions. Experiments on HateMM and MultiHateClip show that our method achieves state-of-the-art performance in the localisation task. Code is available at https://github.com/Multimodal-Intelligence-Lab-MIL/MultiHateLoc.
Deep learning has made remarkable progress in medical image analysis, yet its black-box nature still limits interpretability and clinician trust. Concept-based modeling offers a promising direction for explainable AI by integrating human-understandable concepts. However, existing approaches typically rely on global concept annotations and infer diagnosis based solely on the presence or absence of individual concepts. This oversimplified paradigm ignores the rich relationships among concepts and their causal influence on disease outcomes. To overcome these limitations, we propose the Concept Relationship Embedding Model (CREM) for interpretable medical diagnosis. CREM mirrors coarse-to-fine clinical reasoning by first extracting fine-grained subregional concepts, then explicitly encoding their relationships as a concept interaction graph, and finally performing causal inference between concepts and diagnoses to enable reliable and transparent diagnostic predictions. We evaluate CREM on four public medical imaging benchmarks, where it achieves state-of-the-art performance on both concept recognition and disease classification tasks, while exhibiting improved robustness, label efficiency, and interpretability. Furthermore, we deploy CREM as an interactive web-based demo that allows clinicians to visualize concept activations, trace diagnostic reasoning paths, and iteratively refine concept cues, facilitating effective human-in-the-loop decision-making.