Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with egocentric video to enable context-aware modeling of the driver's attention over time. Furthermore, to tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset that augments four existing driving datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for multimodal cognitive distraction detection. Our codes and CogDrive dataset resources are available at https://github.com/langzhang2000/EyeCue.
论文检索
输入标题、作者或关键词，从 990 篇学术成果中精准定位
Structured output generation is increasingly critical for real-world AI systems, particularly in on-device settings where small language models (0.5B–8B parameters) must produce machine-executable outputs under strict latency and privacy constraints. Although constrained decoding provides formal guarantees of structural validity without retraining, its effectiveness across different tasks, models, and constraint formalisms remains insufficiently understood. We introduce StructureBench, a comprehensive benchmark for structured generation on edge devices, covering JSON and tool-call generation, code synthesis, mathematics, science, and domain-specific languages, and evaluating over 11 on-device language and vision–language models spanning 0.5B–8B parameters. StructureBench systematically compares prompt-based generation with three widely used constrained decoding frameworks—Outlines, XGrammar, and Guidance—and adopts decoupled metrics to separately assess structural validity and semantic correctness. Our experiments show that constrained decoding consistently enforces syntactic validity, but does not reliably improve semantic accuracy and may even degrade performance for smaller models or complex grammars. These findings reveal clear task- and model-dependent boundaries for effective constrained decoding and highlight the need for scenario-aware adaptation in on-device structured generation. We release StructureBench at https://github.com/Str-Ben/StructureBench.
Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute–object compositions by leveraging shared knowledge from seen primitives. While recent works have begun to incorporate test-time adaptation to mitigate performance degradation caused by label-space distribution shifts, they suffer from primitive-level knowledge underutilization and cross-modal semantic asymmetry, hindering robust understanding of novel compositions. To address these issues, we introduce MLDA, a test-time adaptation framework for CZSL that jointly derives multi-level prototypes and performs dynamic vision–language alignment. Specifically, we construct and progressively update composition- and primitive-level prototype sets to accumulate structured semantic knowledge. By augmenting test inputs with diverse visual perspectives and enriched compositional descriptions, MLDA leverages optimal transport to achieve consistent matching of semantically salient information across modalities. Furthermore, composition activation scores are dynamically updated over the test stream to emphasize informative compositions and suppress irrelevant ones. Extensive experiments are conducted to verify the effectiveness of the proposed method. The code is available at https://github.com/keepgoingjkg/MLDA.
Open‑vocabulary video instance segmentation (OV‑VIS) couples spatial‑temporal reasoning with language grounding, yet its adversarial robustness has remained unexplored. We present the Dual-Objective Triggers (DOT), the first transferable attack on OV-VIS that simultaneously exploits the vision–language coupling and temporal coherence. DOT deploys a Dual Semantic Perturbation Module that overlays two complementary triggers: a Semantic Suppression Trigger erases the alignment between the true object and the query, while a Plausible Replacement Trigger steers the tracker toward a phantom trajectory that is visually plausible and text‑consistent. To amplify cross‑model transferability without sacrificing perceptual fidelity, we introduce Phase‑Guided Adversarial Training, which injects perturbations primarily in the phase spectrum while blending amplitudes with clean references. Extensive experiments on four state‑of‑the‑art OV‑VIS implementations demonstrate that DOT reduces mAP by up to 69.3% and raises attack success rate by up to 98%, outperforming the strongest baselines by a factor of 1.6× on average, while maintaining a PSNR of 52.49 dB, thus exposing critical security vulnerabilities and laying a foundation for future research on robust and trustworthy vision–language systems.
Bridging the Objective Gap: A Unified Pre-Training Framework for Few-Shot Medical Image Segmentation
Few-shot medical image segmentation relies on dense, boundary-sensitive prototype matching, yet common pre-training objectives mainly optimize global alignment or reconstruction, creating an objective gap that hurts boundary delineation and increases adaptation cost. This raises the question: how to pre-train representations intrinsically matchable for episodic FSS while requiring minimal adaptation-induced re-organization? We introduce a Drift-Gap diagnostic to quantify intrinsic dense-matching misalignment and adaptation-induced feature drift. Guided by this lens, we propose BOG-PRETRAIN, combining Reliability-Gated Alignment to mitigate noisy report supervision, Semantic-Guided MIM to emphasize boundary-informative regions, and Dual Consistency Regularization to stabilize episodic metric geometry. Across five benchmarks (1/4/16-shot), BOG-PRETRAIN improves mean Dice by +15.5/+15.9/+12.5 points over best priors and reduces mean HD95 by 0.9/4.6/10.1; it achieves the lowest Drift (0.052 vs. 0.155 baseline) and Gap_PT (0.256), with ablations confirming the components' complementarity.
In Partially Cooperative Markov Games (PCMG), agents need to learn effective cooperation under individual rewards, with the key challenge lies in leveraging these rewards to enhance team benefits optimally. Model aggregation (MA) is a promising solution due to its simplicity and efficiency, yet it suffers from instability caused by the volatility of individual policies trained with individual rewards and the non-asymptotic nature of aggregation updates. To address these challenges, we propose Two-Stage Model Aggregation (TSMA), a novel multi-policy MA framework for multi-agent reinforcement learning (MARL) that leverages individual policies to enhance team cooperation. Specifically, to enhance the stability and expressiveness of individual policies, each TSMA agent trains two individual policies (trained using the same individual reward) with parameter-level variation and combines them via adaptive weighted aggregation. The aggregated individual policy is then integrated with the team policy (trained using team reward), with integration weights dynamically adjusted over training. Additionally, L2 regularization is applied during the data-driven training to align their parameter distributions across policies, mitigating instability from non-asymptotic aggregation. As a general framework, TSMA is compatible with standard actor-critic MARL algorithms. We integrate TSMA with MAPPO and MADDPG, and evaluate them on challenging PettingZoo benchmarks, demonstrating significant improvements in PCMG. The code is available at: https://github.com/RL-DLMU/TSMA.
Efficient transfer learning methods for large-scale vision–language models (e.g., CLIP) enable strong few-shot transfer, yet existing adaptation methods follow a fixed fine-tuning paradigm that implicitly assumes a uniform importance of the image and text branches, which has not been systematically studied in image classification. Through extensive analysis, we reveal a Branch Bias issue in vision–language image classification: adapting the image encoder does not always improve performance under out-of-distribution settings. Motivated by this observation, we propose A₃B₂, an Adaptive Asymmetric Adapter that alleviates Branch Bias in few-shot learning. A₃B₂ introduces Uncertainty-Aware Adapter Dampening (UAAD), which automatically suppresses image-branch adaptation when prediction uncertainty is high, enabling soft and data-driven control without manual intervention. Architecturally, A₃B₂ adopts a lightweight asymmetric design inspired by mixture-of-experts with Load Balancing Regularization. Extensive experiments demonstrate that A₃B₂ consistently outperforms 11 competitive prompt- and adapter-based baselines.
With the rapid growth of digital data, real-world applications increasingly involve hierarchical information that combines static attributes with dynamic records. Modeling such heterogeneous data in a unified and generalizable manner remains challenging. Existing approaches often rely on extensive manual design, are tightly coupled to specific data schemas, and typically process static and dynamic attributes in isolation, thereby overlooking their implicit interactions. We propose UniSAGE, a unified framework for modeling data with both static and dynamic attributes. UniSAGE constructs a global attribute graph that represents hierarchical and temporal relationships in a unified structure. To ensure representational consistency, it introduces two orthogonal parameter subspaces that jointly support static aggregation and dynamic reasoning within a shared semantic space. Building on these unified representations, UniSAGE further enables task-specific interaction between static and dynamic attributes via a lightweight hyper-structure mechanism. UniSAGE is fully automated, robust to evolving data schemas, and capable of capturing complex cross-attribute dependencies. Extensive experiments on multiple public benchmarks and a real-world financial behavior dataset demonstrate that UniSAGE consistently outperforms existing methods, achieving performance improvements of over 10% on several tasks. Our code is available at https://github.com/zjunet/UniSAGE.
High-fidelity reconstruction of mel-spectrograms from EEG signals remains a formidable challenge, primarily due to the inherent inter-subject variability of neural patterns and the semantic gap between heterogeneous feature representations of these two modalities. To alleviate both issues, this paper proposes a Hierarchical Feature Fusion Network with bi-phase subject IDentifier modulation (HFFN-ID) for reconstructing mel-spectrograms from EEG signals. The primary improvements of the HFFN-ID framework are from two aspects, a Binary-Phase Subject-ID Modulation (BiPSM) mechanism for explicit subject conditioning and a Condition-guided Hierarchical Fusion (CHF) component for dynamic multi-layer feature synthesis, which respectively aim to address the problems of inter-subject variability in neural patterns and heterogeneous cross-modality feature fusion. Moreover, a speech envelope feature-based pre-training strategy is incorporated to initialize the parameter space, inspired by the shared low-level representations across different speech features. On the SparrKULee dataset, HFFN-ID establishes a new benchmark with a pearson correlation coefficient of 0.0723, representing a substantial 44% relative improvement over existing baseline. Importantly, HFFN-ID is highly efficient, achieving a 38.7% parameter reduction and a 2.5× training speedup. These results highlight the effectiveness of subject-conditioned and hierarchically fused architectures for advancing high-fidelity neural speech decoding.
Open-vocabulary 3D understanding aims to align 3D representations with a unified vision-language semantic space. However, existing methods suffer from the challenge of structural asymmetry caused by sparse observations and holistic geometries. Additionally, the inherent semantic chasm between discrete coordinates and abstract natural language hinders cross-modal mapping. This work introduces SpecBridge, a 3D-2D-Text pre-training framework that leverages CLIP priors as a foundational bridge to connect three modalities by synergizing spectral graph theory with transitive semantic learning. The method consists of two main sub-modules. First, Spectral Eigen-Modal Alignment is presented to construct cross-modal correlations within the intrinsic eigen-spectral space via Laplacian eigendecomposition. By aligning low-frequency geometric harmonics with CLIP-guided observation inference, it maintains structural consistency between 3D geometries and 2D projections. Second, a Transitive Spectral-Semantic Alignment method is developed to establish a 3D→2D→Text propagation chain across the CLIP priors bridge. It distills dense CLIP priors into 3D representations through transitive distillation, effectively mitigating the semantic chasm between geometry and language. Extensive experiments confirm that the proposed SpecBridge demonstrates state-of-the-art performance by overcoming challenges triggered by modality discrepancies.
Conversational Aspect-based Sentiment Quadruple Analysis (DiaASQ) needs to capture the complex interrelationships in multiple rounds of dialogues. Existing methods usually employ simple Graph Convolutional Networks (GCN), which introduce structural noise and fail to consider the temporal sequence of the dialogues, or use standard RoPE, which implicitly captures relative distances in a flat sequence but cannot clearly separate the token-level syntactic order from the utterance-level progression, and may suffer from the Distance Dilution problem. To address these issues, we propose a new framework that combines Thread-Constrained Directed Acyclic Graph (TC-DAG) and Discourse-Aware Rotary Position Embedding (D-RoPE). Specifically, TC-DAG filters out cross-thread noise based on thread constraints, maintains global connectivity through root anchoring, and incorporates the temporal sequence of the dialogues. D-RoPE aligns multi-layer semantics using dual-stream projection and multi-scale frequency signals, captures thread dependencies using tree-like distances, and alleviates the token-level Distance Dilution problem by incorporating utterance-level progressions. Experimental results on two benchmark datasets demonstrate that our framework achieves state-of-the-art performance.
Vision-Language-Action (VLA) models bridge multimodal reasoning with physical control, but adapting them to new tasks with scarce demonstrations remains unreliable. While fine-tuned VLA policies often produce semantically plausible trajectories, failures often arise from unresolved geometric ambiguities, where near-miss actions lead to divergent execution outcomes under limited supervision. We study few-shot VLA adaptation from a generation-selection perspective and propose a novel framework, VGAS (Value-Guided Action-chunk Selection). It performs inference-time best-of-N selection to identify action chunks that are both semantically faithful and geometrically precise. Specifically, VGAS employs a finetuned VLA as a high-recall proposal generator and introduces Q-Chunk-Former, a geometrically grounded Transformer critic to resolve fine-grained geometric ambiguities. In addition, we propose Explicit Geometric Regularization (EGR), which shapes a discriminative value landscape to preserve action ranking resolution among near-miss candidates while mitigating value instability under scarce supervision. Experiments and theoretical analysis demonstrate that VGAS consistently improves success rates and robustness under limited demonstrations and distribution shifts. Our code is available at https://github.com/Jyugo-15/VGAS
Federated optimization under data heterogeneity presents a significant challenge, often leading to suboptimal model performance. While numerous methods aim to replicate the ideal performance of centralized training, they frequently fall short in highly heterogeneous settings. In this paper, we introduce HaFedHo, an adaptive objective rectification method that harmonizes local training with the ideal data-centralized objective, requiring minimal modifications to the standard federated learning framework. HaFedHo operates by first decoupling the centralized objective and then employing a dynamic Taylor series expansion to accurately estimate the global objective for each client. Our theoretical analysis shows that the estimation error provably converges to zero as training progresses. Furthermore, extensive experiments on real-world datasets demonstrate that HaFedHo surpasses state-of-the-art methods, including SCAFFOLD, MimeLite, and FedDyn, in both test accuracy and communication efficiency. Notably, HaFedHo maintains its superior performance even with a client participation rate as low as $0.2\%$ in severely heterogeneous environments.
While deep learning has significantly enhanced robotic perception in agriculture, autonomous decision-making in dense and occluded environments remains a persistent challenge. This paper proposes a VR-based expert motion capture framework to bridge this gap by integrating high-fidelity virtual environments with human expertise. By reconstructing a realistic tomato greenhouse in Unity and utilizing 27-point motion capture suits, we captured multi-dimensional expert demonstration data encompassing trajectories, postures, and velocities. In the VR environment, ``hard cases" featuring frequent collisions can be selectively generated at a higher frequency, allowing experts to provide concentrated demonstrations of specialized postures that markedly accelerate algorithmic learning efficiency. Crucially, the virtual framework enables multiple experts to repetitively harvest identical targets to match optimal trajectories—a process precluded in reality by the destructive nature of harvesting. Moreover, it bypasses biological growth and seasonal constraints, facilitating continuous, high-frequency experimentation independent of crop maturation. These expert data were subsequently integrated into a multi-task reinforcement learning model as experience replay and reward templates. Experimental results demonstrate that our expert-driven algorithm significantly outperforms baseline methods in harvesting efficiency, collision avoidance, and stability, establishing VR-guided motion capture as a robust and scalable paradigm for advancing agricultural robotics.
Explainable Recommendation (ER) aims to enhance recommendation transparency and prediction accuracy by providing faithful and persuasive explanations. However, Multi-Modal Multi-Interest Explainable Recommendation (MMER) is particularly challenging in two aspects: effectively utilizing diverse multi-modal information to construct reliable semantic evidence and precisely identifying the most relevant user interest to generate faithful explanations. Most previous methods fail to effectively exploit visual information to provide reliable evidence and primarily rely on a single unified representation of user preferences, making it difficult to distinguish diverse user interests and generate faithful explanations. To fill this gap, we propose Joint Multi-Modal Multi-Interest Profiling and Preference-Grounded Reasoning (PRIME) for solving the MMER problem. Specifically, PRIME first organizes items into interest-aware clusters to facilitate interest disentanglement. Based on these clusters, it constructs explicit multi-modal multi-interest user profiles and a comprehensive global item profile to capture fine-grained preferences and distinguish diverse user interests. Furthermore, PRIME performs preference-grounded reasoning by retrieving the most relevant user interest for each item, enabling faithful explanations and accurate predictions. Our experiments on three real-world datasets demonstrate that PRIME outperforms the state-of-the-art methods.
Recent generative AI renders multimodal misinformation structurally harder to detect, making reliable detection dependent on semantic verification grounded in verifiable evidence. However, current benchmarks often fail to isolate true semantic checking from superficial shortcut exploitation. We introduce Falsdo, a diagnostic benchmark designed to make robustness and auditability identifiable. Constructed via a heterogeneous web-mining pipeline, Falsdo provides instruction-conditioned grounding and formalizes a counterfactual artifact-control protocol. This stress test reveals a critical failure: when low-level generation traces are equalized, detector performance degrades, indicating reliance on artifacts rather than robust verification. To enable reproducible diagnosis, we propose DREA, a failure-aligned evidence-auditing baseline. DREA specifies evidence acquisition with explicit noise control and implements constrained channel-wise auditing with reliability-aware late fusion. Experiments demonstrate that Falsdo reliably exposes shortcut dependence, while DREA improves instruction-level grounding (Joint-F1) and minimizes degradation under artifact-controlled settings.
Knowledge distillation has become a prevalent technique for deploying efficient recommender systems, enabling lightweight student models to approximate the performance of larger teachers. However, we identify a critical issue: distillation systematically amplifies popularity bias, as student models inherit and intensify the popularity-driven shortcuts encoded in teachers trained on interaction data dominated by popular items. To address this limitation, we propose GUIDE (Geodesic aware Unbiased Instructive Distillation with Experts), a collaborative distillation framework that incorporates domain-specific debiasing experts alongside the global teacher. GUIDE tackles two key challenges in this paradigm. First, for expert routing, we introduce Spherical Expert Alignment, which conducts expert-student matching on the spherical manifold with geodesic distance optimization, eliminating magnitude-induced bias and ensuring stable gradient flow. Second, for context fusion, we design a Meta-Debiasing Gate that dynamically arbitrates teacher-expert influence based on real-time user-item context through end-to-end meta-learning. Extensive experiments on multiple real-world datasets demonstrate that GUIDE significantly mitigates popularity bias while preserving recommendation accuracy, with state-of-the-art trade-offs among efficiency, accuracy, and fairness. The code and data are available at:https://github.29com/zx19971219/GUIDE.
Hand Object Interaction (HOI) generation provides an efficient solution for virtual reality simulation and embodied AI deployment.Recent studies have explored instruction-driven HOI synthesis, yet they overlooked the fine-grained interactive contact and struggled with robust generalization in data-scarce scenarios. In addition, existing datasets are frequently constrained with limited object categories and monotonous motion patterns. To address these limitations, we first integrate a large-scale HOI dataset, namely HOI-X, which contains diverse objects, rich motion patterns, and part-level contact annotations. Along with HOI-X, we propose A3fford-HOI, a fine-grained and generalizable Hand Object Interaction framework with Anatomy-Aligned Affordance Disentanglement. Specifically, we define a Disentangle-2-Interact paradigm in A3fford-HOI. Considering the various interactions of distinct hand anatomical structures, in the Disentangle stage, we devise an Anatomy-Aligned Affordance Predictor, which leverages the external plausible physical priors in 3D multimodal large language models for affordance disentanglement prediction. Then, the disentangled affordance is employed into the Interact stage, i.e. Affordance Driven Interactor. The interactor integrates textual instructions, object geometric features, and the external knowledge enhanced (MLLMs) affordance via a Diffusion Transformer (DiT) for generalizable HOI generation. Extensive experiments demonstrate that A3fford-HOI maintains superior generation quality and contact plausibility for fine-grained and generalizable hand object interaction.
Recommendation system technology faces the fundamental challenge of data sparsity. Although incorporating auxiliary textual data works to enrich interactions, lengthy and unstructured item descriptions often fail to depict user interests, as they primarily describe item attributes. Furthermore, the sparse interactions provide incomplete interest patterns, leading to biased recommendations. To address these issues, this work proposes an LLM-summarized interest distillation (SID) model to enrich interactions by interest summarization and retrieval augmentation. Interest summarization leverages large language model to improve user interest descriptions and filters out interest-irrelevant details, generating compact, user-centric interest texts. Retrieval augmentation utilizes semantic neighbors as multiple teachers to adaptively transfer their multimodal collaborative knowledge to augment behaviorally sparse student users, thereby enriching interest learning for recommendation. It enhances interactions both qualitatively, through improved interest descriptions, and quantitatively, through expanded semantic neighbors. Extensive experiments show the superior performance of SID over the state-of-the-art models, demonstrating its effectiveness of interest summarization and retrieval augmentation to improve recommendations.
Accurate traffic flow prediction is fundamental to Intelligent Transportation Systems (ITS). However, traffic dynamics exhibit inherent multi-scale heterogeneity, where stable global trends are often masked by stochastic local fluctuations. Existing methods struggle to reconcile these conflicting resolutions, leading to sub-optimal forecasting. To address this, we propose the Multi-Scale Spatial-Temporal Diffusion Transformer (MS-STDT). Deviating from previous diffusion approaches that treat generation as a monolithic task, we reframe the forward diffusion process as an intrinsic temporal coarse-graining operation. By leveraging the data's multi-scale hierarchy as a structural anchor, we introduce a coarse-to-fine latent guidance strategy that enables the model to reconstruct stable global trends before refining fine-grained details. This ensures physical consistency and generative stability without requiring external labels. Extensive experiments across six real-world datasets confirm that MS-STDT performs the best, demonstrating significant improvements in predictive accuracy and zero-shot robustness against sensor failures. We provide code and data at https://github.com/ZetaoLiPhD/MS-STDT.