Collaborative human-object interaction shows dynamic and complex movements that require mutual anticipation and continuous adjustment between participants and the shared object. Understanding and modeling such collaborative multi-human object interaction (MHOI) scenarios requires high-quality data acquisition as a foundational step, however, this is a challenging task due to the inherent complexity of MHOI scenarios where human-human and human-object interactions occur simultaneously. Such complexity leads to noisy MHOI captures characterized by several artifacts: contact misalignment between hands and objects, motion jitter and temporal inconsistencies in the captured sequences, and missing or incomplete finger-level articulation details. To address these challenges, we present MOCHI (MOtion Enhancement of Collaborative Human-object Interactions), a two-stage framework for enhancing noisy MHOI data. Our approach first generates physically plausible hand grasps through optimization from noisy body input, producing grasps that are both physically plausible and semantically consistent with the body pose, where these optimized grasps are extended into complete hand-object interaction sequences. Consequently, the full-body motion for all participants are refined through a diffusion-based noise optimization framework that uses single-person motion priors. During the optimization process, we introduce optimization objectives to encode human-object and human-human interaction information within these single-person priors. Experimental results demonstrate the effectiveness of our pipeline across diverse MHOI data, either acquired by existing capture methods or synthesized by generative models. We further show robustness of our system across varying numbers of participants and types of interactions, and demonstrate various applications including keyframe-based MHOI creation and data augmentation through varying object geometries.
论文检索
输入标题、作者或关键词，从 13,734 篇学术成果中精准定位
Mixed Reality (MR) headsets promise a future of immersive telepresence where virtual humans blend indistinguishably into their real or virtual surroundings. Achieving this vision requires a method capable of capturing a user's motion, estimating their appearance under novel lighting, and understanding the surrounding environment - all from the constrained viewpoint of a head-mounted display (HMD). Existing approaches treat these as isolated problems: they either focus on driving avatars with baked-in lighting or rely on complex studio setups for relighting. In this paper, we present EgoRelight, a holistic framework for egocentric telepresence that simultaneously captures full-body human performance, synthesizes photorealistic and relightable appearance, and estimates high dynamic range (HDR) environment maps from a single HMD. First, to ensure accurate motion and surface reconstruction, we propose an egocentric perception module that leverages stereo down-facing cameras to extract dense depth maps, which serve as robust geometric control signals to drive a mesh-based avatar. Second, we introduce a novel neural appearance model that learns to synthesize view-dependent specular and view-independent diffuse shading separately. By employing a specialized ray-sampling strategy, our model generalizes to unseen illumination without relying on restrictive analytical BRDF priors. Third, we enable seamless avatar integration into the physical world via a test-time inverse rendering process, which recovers an HDR environment map by matching the pre-trained avatar's appearance to the live egocentric camera observations. We demonstrate the efficiency of our system through a social telepresence application, where remote users are coherently relit according to their local physical environment. Extensive experiments show that our individual components and the integrated system significantly outperform the best combinations of state-of-the-art baselines in both geometric accuracy and rendering as well as relighting fidelity. Further details can be found on our project page1.
We revisit the computation of 3D generalized winding numbers, a useful measure for inside-outside classification on triangle meshes with gaps, self-intersections, and open boundaries. At the core of our new method is an analytical reduction of the surface integral that defines the winding number, resulting in a single ray-mesh intersection test and an elementary sum over boundary edges per evaluation. This construction is orders of magnitude more efficient than the state of the art in practice, which we show in an extensive performance benchmark. Conveniently, the method also reduces to the best-available asymptotic complexity in the worst case, and it introduces no approximations apart from floating-point errors. Our algorithm is conceptually simple to understand, straightforward to implement and debug, and it works reliably even on extremely noisy and corrupt input geometry.
Selection is a core operation in interactive image editing, enabling tasks such as composition or manipulation. To be practically useful, a user should be able to specify and disambiguate the desired selection region through either text- or click-based interactions, and the system should support selecting not only objects but also other criteria, such as materials. Material-based selection can be particularly valuable for tasks like re-texturing surfaces or consistently editing all instances of a specific material in a scene. However, existing vision–language-model (VLM) based selection methods are largely object-centric and typically support only a single interaction modality, limiting their applicability in real editing workflows. In this work, we thus present Mask Any Object And Material (MAOAM), a unified selection framework that enables precise object- and material-level selection across both text- and click-based interactions. MAOAM leverages a VLM with a segmentation head to produce pixel-accurate masks from user prompts: the VLM interprets the user’s selection intent — object- or material-level — and encodes visual entities, attributes, and spatial relations, while the segmentation head decodes the VLM’s output token into a mask. A key challenge is that material selection datasets with text annotations are unavailable. We therefore propose a scalable data generation pipeline: we collect real and synthetic images with material masks, then leverage VLMs to generate material descriptions with rich visual-semantic information. Using the generated data, we train MAOAM with a multi-task objective over click- and text-based selection, along with an auxiliary VQA task derived from the material descriptions to facilitate deeper material understanding. Despite being trained with uni-modal prompts, our model exhibits an emergent improvement in selection quality when combining text and clicks at inference time, enabling more flexible image editing workflows. Experiments demonstrate accurate and coherent selections across diverse objects, materials, and interaction scenarios, highlighting robustness in practice.
As instruction-based editing models and multimodal large language models continue to advance, a wide range of image editing tasks have become feasible. However, achieving precise and consistent geometric image editing, such as translating, scaling and rotating in 3D space, remains a major challenge. In this work, we introduce BoxCtrl, a 3D-aware visual prompting framework to address this problem. Unlike text-only or coarse 2D-guided approaches, our method introduces informative RGB 3D bounding boxes projected onto 2D images as a visual prompting. Each box is outlined and its three orthogonal faces are individually painted with different RGB colors, which simultaneously encode its position, size and orientation, providing a compact and intuitive in-context visual example. The key to BoxCtrl’s success lies in its well-designed bounding boxes, which serve as visual prompting that decouple geometric control from appearance control. This design enables the model to learn consistent correspondences between faces with the same color in the latent space, leading to a more precise understanding of geometric intention and more accurate editing result. We introduce a two-stage training paradigm comprising Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL). To address the scarcity of paired training data, we first construct a large-scale synthetic dataset for the SFT stage, which equips the model with fundamental editing capabilities. However, the generalization of the SFT model is constrained by the synthetic-to-real domain gap. To bridge this, we incorporate an online RL stage leveraging unpaired real-world data. Guided by a comprehensive reward function evaluating both geometric accuracy and visual fidelity, our SFT-RL strategy significantly enhances geometric precision while maintaining photorealistic quality. Extensive experiments demonstrate that BoxCtrl achieves state-of-the-art performance across translation, rotation, scaling, and composite editing tasks. Our code is available at https://github.com/beaglew/BoxCtrl.
Developments in display technology mean that it may soon be possible to create stereoscopic 3-D imagery that cannot be distinguished from its real-world counterpart, i.e., it could pass the so-called Visual Turing Test. To measure how far current display technologies are from this ultimate goal, we built a custom ultra-realistic display — a high dynamic range (HDR) stereoscope, with accurate color and geometric calibration. Then, we used the display to measure how the reduction of display resolution and dynamic range (contrast) affects the sense of realism, relative to a real scene. These two display capabilities have improved greatly in recent years, and it is useful to understand what benefits further development might bring. The display resolution measurements showed that the realism of presented scenes starts dropping rapidly below ∼ 60 pixels per degree, and the gains diminish above that resolution. Our contrast measurement showed a linear reduction in realism as the contrast was reduced. Furthermore, we found that the observers could discriminate between real and virtual scenes in less than half of the trials (when adjusted for the guess rate), demonstrating the ultra-realistic capabilities of our display. Our results provide a clear guide to potential gains in realism, and can inform display design and HDR tone mapping.
Despite rapid advances in multimodal large language models, agricultural applications remain constrained by the lack of multilingual speech data, unified multimodal architectures, and comprehensive evaluation benchmarks. To address these challenges, we present AgriGPT-Omni, an agricultural omni-framework that integrates speech, vision, and text in a unified framework.(1) First, we construct a scalable data synthesis and collection pipeline that converts agricultural texts and images into training data, resulting in the largest agricultural speech dataset to date, including 492K synthetic and 1.4K real speech samples across six languages.(2) Second, based on this, we train the first agricultural Omni-model via a three-stage paradigm: textual knowledge injection, progressive multimodal alignment, and GRPO-based reinforcement learning, enabling unified reasoning across languages and modalities.(3) We further propose AgriBench-Omni-2K, the first tri-modal benchmark for agriculture, covering diverse speech–vision–text tasks and multilingual slices, with standardized protocols and reproducible tools. Experiments show that AgriGPT-Omni significantly outperforms general-purpose baselines on multilingual and multimodal reasoning as well as real-world speech understanding.
A lesson plan (LP) is a structured guide outlining instructional objectives, methods, and assessments to ensure organized learning. However, existing LP creations are often time-consuming, inconsistent in structure, and lack pedagogical mechanisms for real-time adaptation to diverse learner needs. To address these issues, we propose a co llaborative multi-role a gent framework called COMA for automatic LP generation. COMA formulates LP generation as a collaborative workflow among multiple LLM agents with distinct pedagogical expertise: (1) the novice agent that represents a novice teacher possesses an overarching understanding of the intended lesson flow but demonstrates limited precision in implementing the specific instructional actions; (2) the veteran agent that represents an experienced teacher demonstrates deep familiarity with the curriculum, textbooks, and the knowledge components embedded in each unit; and (3) the master agent that represents a pedagogical expert exhibits a well-developed and confident grasp of lesson progression, with the ability to design, adapt, and implement specific instructional actions effectively and responsively. Through an iterative workflow, these agents collaboratively refine LP quality. Comprehensive experiments across five subjects, using expert-designed metrics, demonstrate that COMA significantly outperforms state-of-the-art methods, producing lesson plans with superior quality, coherence, and pedagogical alignment. Our framework offers a robust solution for generating deployable instructional content at scale. Data and code are available at https://github.com/ai4ed/COMA-LessonPlan.
As web platforms evolve towards greater personalization and emotional complexity, conversational agents must transcend superficial empathy to demonstrate identity-aware emotional reasoning. However, existing systems face two limitations: (1) reliance on situation-centric datasets lacking persistent user identity, which hampers the capture of personalized affective nuances; and (2) dependence on opaque, coarse reward signals that hinder development of verifiable empathetic reasoning. To address these gaps, we introduce KardiaBench, a large-scale user-grounded benchmark comprising 178,080 QA pairs across 22,080 multi-turn conversations anchored to 671 real-world profiles. The dataset is constructed via a model-in-the-loop pipeline with iterative rubric-guided refinement to ensure psychological plausibility and persona consistency. This progressive empathy pipeline that integrates user comprehension, contextual reasoning, and emotion perception into conversations, followed by iterative critique and rubric-based refinement to ensure psychological plausibility, emotional fidelity, and persona consistency. Building on this, we propose Kardia-R1, a framework that trains models for interpretable, stepwise empathetic cognition. Kardia-R1 leverages Rubric-as-Judge Empathetic Reinforcement Learning (Rubric-ERL), a GRPO-based method that uses explainable, human-aligned rubric rewards to tightly couple user understanding, emotional inference, and supportive response generation. Extensive experiments across four LLM backbones demonstrate that Kardia-R1 consistently outperforms other methods in emotion accuracy, empathy, relevance, persona consistency, and safety.
The rapid proliferation of VTubers --- digital avatars controlled and voiced by human actors (Nakanohito) --- has created a lucrative and popular entertainment ecosystem. However, the prevailing industry model, where corporations retain ownership of the VTuber persona while the Nakanohito bears the immense pressure of dual-identity management, exposes the Nakanohito to significant vulnerabilities, including burnout, harassment, and precarious labor conditions. When these pressures become untenable, the Nakanohito may terminate their contracts and later debut with a new persona, a process known as ''reincarnation''. This phenomenon, a rising concern in the industry, inflicts substantial losses on the Nakanohito, agencies, and audiences alike. Understanding the quantitative fallout of reincarnation is crucial for mitigating this damage and fostering a more sustainable industry. To address this gap, we conduct the first large-scale empirical study of VTuber reincarnation, analyzing 12 significant cases using a comprehensive dataset of 728K livestream sessions and 4.5B viewer interaction records. Our results suggest reincarnation significantly damages a Nakanohito's career, leading to a decline in audience and financial support, an increase in harassment, and negative repercussions for the wider VTuber industry. Overall, these insights carry immediate implications for mitigating the significant professional and personal costs of the reincarnation, and fostering a healthier and more equitable VTuber ecosystem.
Among populations with limited literacy in emerging digital markets, the adoption of mobile phones, combined with comprehension barriers and poor cybersecurity hygiene, has created hidden privacy risks. This paper examines how informed consent is often abused by predatory financial applications, leading to financial scams that disproportionately affect users with low literacy. We focus on predatory loan, gambling, and trading apps, analyzing a dataset of 50 Google Play Store apps to measure how many omit or obfuscate critical privacy disclosures. We also evaluate comprehension gaps among users with low literacy via a targeted user study and assess whether Large Language Model (LLM)-generated summaries, translations, and visual cues can improve consent clarity. Our findings show that 85% of study participants did not understand basic app permissions, underscoring the urgent need for stronger regulatory oversight and scalable LLM-driven privacy-literacy tools.
The shortage of mental health professionals has driven the web to become a primary avenue for accessible psychological support. While Large Language Models (LLMs) offer promise for scalable web-based counseling, existing approaches often lack emotional understanding, adaptive strategies, and long-term memory. These limitations pose risks to digital well-being, as disjointed interactions can fail to support vulnerable users effectively. To address these gaps, we introduce TheraMind, a strategic and adaptive agent designed for trustworthy online longitudinal counseling. The cornerstone of TheraMind is a novel dual-loop architecture that decouples the complex counseling process into an Intra-Session Loop for tactical dialogue management and a Cross-Session Loop for strategic therapeutic planning. The Intra-Session Loop perceives the patient's emotional state to dynamically select response strategies while leveraging cross-session memory to ensure continuity. Crucially, the Cross-Session Loop empowers the agent with long-term adaptability by evaluating the efficacy of the applied therapy after each session and adjusting the method for subsequent interactions. We validate our approach in a high-fidelity simulation environment grounded in real clinical cases. Extensive evaluations show that TheraMind outperforms other methods, especially on multi-session metrics like Coherence, Flexibility, and Therapeutic Attunement, validating the effectiveness of its dual-loop design in emulating strategic, adaptive, and longitudinal therapeutic behavior. The code is publicly available at https://github.com/Emo-gml/TheraMind.
Major social media platforms increasingly adopt community-based fact-checking to address misinformation on their platforms. While previous research has largely focused on its effect on engagement (e. g., reposts), an understanding of how fact-checks affect a user's follower base is missing. In this study, we employ quasi-experimental methods to causally assess whether users lose followers after their posts are corrected via community fact-checks. Based on time-series data on follower counts for N = 4391 community fact-checked posts from X, we find that community fact-checks do not lead to meaningful declines in the follower counts of users who post misleading content. This suggests that followers of spreaders of misleading posts tend to remain loyal and do not view community fact-checks as a sufficient reason to disengage. Our findings underscore the need for complementary interventions to more effectively disincentivize the production of misinformation on social media.
Accurate forecasting of Avian Influenza Virus (AIV) outbreaks within wild bird populations necessitates models that account for complex, multi-scale transmission patterns driven by diverse factors. While conventional spatiotemporal epidemic models are robust for human-centric diseases, they rely on spatial homophily and diffusive transmission between geographic regions. This simplification is incomplete for AIV as it neglects valuable genomic information critical for capturing dynamics like high-frequency reassortment and lineage turnover at the case level (e.g., genetic descent across regions), which are essential for understanding AIV spread. To address these limitations, we systematically formulate the AIV forecasting problem and propose BLUE (bi-layer genomic-aware heterogeneous graph fusion pipeline). This pipeline integrates genetic, spatial, and ecological data to achieve highly accurate outbreak forecasting. It 1) defines a multi-layered graph structure incorporating information from diverse sources and multiple layers (case and location), 2) applies cross-relation smoothing to smooth information flow across edge types, 3) performs graph fusion that preserves critical structural patterns backed by theoretical spectral guarantees, and 4) forecasts future outbreaks using an autoregressive graph sequence model to capture transmission dynamics. To support research, we release the Avian-US dataset, which provides comprehensive genetic, spatial, and ecological data on US avian influenza outbreaks. BLUE demonstrates superior performance over existing baselines, highlighting the efficacy of integrating multi-layer information for infectious disease forecasting. The code is available at: https://github.com/cruiseresearchgroup/BLUE.
Deep learning has made remarkable progress in medical image analysis, yet its black-box nature still limits interpretability and clinician trust. Concept-based modeling offers a promising direction for explainable AI by integrating human-understandable concepts. However, existing approaches typically rely on global concept annotations and infer diagnosis based solely on the presence or absence of individual concepts. This oversimplified paradigm ignores the rich relationships among concepts and their causal influence on disease outcomes. To overcome these limitations, we propose the Concept Relationship Embedding Model (CREM) for interpretable medical diagnosis. CREM mirrors coarse-to-fine clinical reasoning by first extracting fine-grained subregional concepts, then explicitly encoding their relationships as a concept interaction graph, and finally performing causal inference between concepts and diagnoses to enable reliable and transparent diagnostic predictions. We evaluate CREM on four public medical imaging benchmarks, where it achieves state-of-the-art performance on both concept recognition and disease classification tasks, while exhibiting improved robustness, label efficiency, and interpretability. Furthermore, we deploy CREM as an interactive web-based demo that allows clinicians to visualize concept activations, trace diagnostic reasoning paths, and iteratively refine concept cues, facilitating effective human-in-the-loop decision-making.
Generative Recommendation (GR) has become a promising end-to-end approach with high FLOPS utilization for resource-efficient recommendation. Despite the effectiveness, we show that current GR models suffer from a critical bias amplification issue, where token-level bias escalates as token generation progresses, ultimately limiting the recommendation diversity and hurting the user experience. By comparing against the key factor behind the success of traditional multi-stage pipelines, we reveal two limitations in GR that can amplify the bias: homogeneous reliance on the encoded history, and fixed computational budgets that prevent deeper user preference understanding. To combat the bias amplification issue, it is crucial for GR to 1) incorporate more heterogeneous information, and 2) allocate greater computational resources at each token generation step. To this end, we propose CARE, a simple yet effective cascaded reasoning framework for debiased GR. To incorporate heterogeneous information, we introduce a progressive history encoding mechanism, which progressively incorporates increasingly fine-grained history information as the generation process advances. To allocate more computations, we propose a query-anchored reasoning mechanism, which seeks to perform a deeper understanding of historical information through parallel reasoning steps. We instantiate CARE on three GR backbones. Empirical results on four datasets show the superiority of CARE in recommendation accuracy, diversity, efficiency, and promising scalability.
Understanding experienced segregation in urban daily life is crucial for addressing societal inequalities and fostering inclusivity. The abundance of user-generated reviews on social media encapsulates nuanced perceptions and feelings associated with different places, offering rich insights into segregation. However, leveraging this data poses significant challenges due to its vast volume, ambiguity, and confluence of diverse perspectives. To tackle these challenges, we propose a novel Large Language Model (LLM) Agent to automate online review mining for segregation prediction. Specifically, we propose a reflective LLM coder to digest social media content into insights consistent with real-world feedback, and eventually produce a codebook capturing key dimensions that signal segregation experience, such as cultural resonance and appeal, accessibility and convenience, and community engagement and local involvement. Guided by the codebook, LLMs can generate both informative review summaries and ratings for segregation prediction. Moreover, we design a REasoning-and-EMbedding (RE'EM) framework, which combines the reasoning and embedding capabilities of language models to integrate multi-channel features for segregation prediction. Experiments on real-world data demonstrate that our agent substantially improves prediction accuracy, with a 22.79% elevation in R2 and a 9.33% reduction in MSE. The derived codebook is generalizable across three different cities, consistently improving prediction accuracy. Moreover, our user study confirms that the codebook-guided summaries provide cognitive gains for human participants in perceiving places of interest(POIs)' social inclusiveness. Our study marks an important step toward understanding implicit social barriers and inequalities, demonstrating the great potential of promoting social inclusiveness with Web technology.