We present a data-driven approach for physics-based, muscle-driven dexterous control that enables musculoskeletal hands to perform precise piano playing for novel pieces of music outside the reference dataset. Our approach combines high-frequency muscle-level control with low-frequency latent-space coordination in a hierarchical architecture. At the low level, general single-hand policies are trained via reinforcement learning to generate dynamic muscle-tendon activations while tracking trajectories from a large reference motion dataset. The resulting tracking policies are then distilled into variational autoencoder (VAE) models, yielding smooth and structured latent spaces that abstract away low-level muscle dynamics. For the high level, we train piece-specific policies to operate in this latent space, coordinating bimanual motions based on specific goals, denoted by note events extracted from given musical scores, to synthesize performances beyond the reference data. High-level control is formulated as a decentralized multiagent reinforcement learning problem combined with adversarial learning for motion imitation. In addition, we present an enhanced musculoskeletal hand model that supports fine control of fingers for accurate low-level motion tracking and diverse high-level motion synthesis. We evaluate the control pipeline of our approach on a diverse piano repertoire spanning multiple musical styles and technical demands. Results demonstrate that our approach can synthesize coordinated bimanual motions with accurate key presses, and achieve the state-of-the-art performance of piano playing in physics-based dexterous control, while generalizing to sheet music that is not presented in the reference dataset. We also show that our musculoskeletal hand model demonstrates superior biomechanical stability and tracking precision compared to the existing model, and validate that our musculoskeletal hand model and muscle-driven controller can generate physiologically plausible activation patterns that align with human electromyography (EMG) recordings when subjects perform multiple tasks.
论文检索
输入标题、作者或关键词，从 2,394 篇学术成果中精准定位
Synthesizing realistic 3D indoor scenes remains challenging due to data scarcity and the difficulty of simultaneously enforcing global architectural constraints and local semantic consistency. Existing approaches often overlook structural boundaries or rely on fully connected relation graphs that introduce redundant generation errors. Inspired by human design cognition, we present CasLayout, a cascaded diffusion framework that decomposes the joint scene generation task into four conditional sub-stages with explicit physical and semantic roles: (1) predicting furniture quantity and categories, (2) refining object sizes and feature embeddings, (3) modeling spatial relationships in a latent space, and (4) generating Oriented Bounding Boxes (OBBs). This decoupled architecture reduces data requirements and enables flexible integration of Large Language Models (LLMs) and Vision Language Models (VLMs) for zero-shot tasks such as image-to-scene generation. To maintain physical validity within complex floor plans, we explicitly model building elements (e.g., walls, doors, and windows) as conditional constraints. Furthermore, to address the high entropy of dense relation graphs, we introduce a sparse relation graph formulation aligned with human spatial descriptions. By encoding these sparse graphs into a compact latent space using a bidirectional Variational Autoencoder (VAE), the proposed framework provides enhanced relational controllability, allowing generated layouts to better respect functional organization. Experiments demonstrate that CasLayout achieves state-of-the-art performance in fidelity and diversity while enabling improved controllability in practical applications.
The fundamental limitation of traditional strand-based modeling is not simply data scarcity, but the ill-posedness of inferring complex 3D fields from 2D imagery without structural constraints. This unconstrained regression leads to catastrophic failures in resolving both global occlusion (e.g., in ponytails) and local directionality (e.g., in curls), resulting in over-smoothed, plausible-but-incorrect geometries. To resolve this, we integrate the strong geometric priors of Large Reconstruction Models (LRMs) into the strand generation pipeline. Using the LRM mesh as a structural anchor, we employ a novel Dual Orientation AutoEncoder to lift coarse geometry into high-fidelity strands. By resolving vector field singularities through latent-space optimization and surface-guided refinement, our method effectively disentangles complex topological structures, setting a new benchmark for robustness and accuracy in hair reconstruction.
Generative latent diffusion models (LDMs) have been extensively applied in various fields yet underperform in time-series prediction. Therefore, We propose the Re-Diffusion model, a latent diffusion approach that generates backbone residuals specifically tailored for time-series forecasting. The model comprises a variational autoencoder that compresses the residuals between the actual future values and the predictions from the backbone into latent space. It also includes a conditional diffusion generator to forecast the potential distribution of these residuals. Our findings reveal that this latent-space methodology particularly enhances existing backbone predictors, by effectively reducing prediction bias through an advanced estimation of complex error distributions. While previous diffusion-based models tend to struggle with long-term forecasting, Re-Diffusion integrates the strengths of diffusion methods, leading to improvements in long-term predictions. Our experimental results indicate that the Re-Diffusion model achieves a 10% promotion over state-of-art predictors, marking a significant advancement in the field of time-series forecasting.
Multimodal information extraction (MIE) constitutes a set of essential tasks aimed at extracting structural information from Web texts with integrating images, to facilitate the structural construction of Web-based semantic knowledge. To address the expanding category set including newly emerging entity types or relations on websites, prior research proposed the zero-shot MIE (ZS-MIE) task which aims to extract unseen structural knowledge with textual and visual modalities. However, the ZS-MIE models are limited to recognizing the samples that fall within the unseen category set, and they struggle to deal with real-world scenarios that encompass both seen and unseen categories. The shortcomings of existing methods can be ascribed to two main aspects. On one hand, these methods construct representations of samples and categories within Euclidean space, failing to capture the hierarchical semantic relationships between the two modalities within a sample and their corresponding category prototypes. On the other hand, there is a notable gap in the distribution of semantic similarity between seen and unseen category sets, which impacts the generative capability of the ZS-MIE models. To overcome the above disadvantages, we delve into the generalized zero-shot MIE (GZS-MIE) task and propose the hyperbolic multimodal generative representation learning framework (HMGRL). The variational information bottleneck and autoencoder networks are reconstructed with hyperbolic space for modeling the multi-level hierarchical semantic correlations among samples and prototypes. Furthermore, the proposed model is trained with the unseen samples generated by the decoder, and we introduce the semantic similarity distribution alignment loss to enhance the model's generalization performance. Experimental evaluations on two benchmark datasets underscore the superiority of HMGRL compared to existing baseline methods.
Search and Recommendation (S&R) are core information access channels on modern multi-scenario platforms. Existing joint S&R models face two critical challenges: (1) cross-scenario interest inconsistency, failing to unify explicit search intent (queries) and implicit recommendation intent (behavioral interactions) into coherent user interest representations; (2) severe S&R trade-off, where enhancing one task degrades the other due to static knowledge sharing and unbalanced feature utilization. To address these issues, we propose MinSAR , a novel framework focusing on cross-S&R user interest consistency. It integrates two key innovations: a Unified Interest Generation (UIG) module using Vector Quantized-Variational Autoencoder (VQ-VAE) to fuse long-term user preferences (via a user-specific memory network) and dynamic short-term contextual behaviors, generating compact cross-scenario latent representations that bridge explicit and implicit intents. Additionally, an Interest-Guided Attention Expert Network replaces static multi-task gating with intent-aware weight allocation. Guided by UIG's unified interest, it dynamically balances cross-S&R shared knowledge and task-specific expertise (semantic matching for search, collaborative filtering for recommendation), mitigating inter-task conflicts. Extensive experiments on two real-world datasets (KuaiSAR and Amazon Kindle Store) against 13 baselines show MinSAR outperforms state-of-the-art joint S&R models. Further analysis confirms its ability to eliminate the S&R performance trade-off.
Large Language Model (LLM)-based agentic systems have shown strong capabilities across various tasks. However, existing multi-agent frameworks often rely on static or task-level workflows, which either over-process simple queries or underperform on complex ones, while also neglecting the efficiency-performance trade-offs across heterogeneous LLMs. To address these limitations, we propose Difficulty-Aware Agentic Orchestration (DAAO), which can dynamically generate query-specific multi-agent workflows guided by predicted query difficulty. DAAO comprises three interdependent modules: a variational autoencoder (VAE) for difficulty estimation, a modular operator allocator, and a cost- and performance-aware LLM router. A self-adjusting policy updates difficulty estimates based on workflow success, enabling simpler workflows for easy queries and more complex strategies for harder ones. Experiments on six benchmarks demonstrate that DAAO surpasses prior multi-agent systems in both accuracy and inference efficiency, validating its effectiveness for adaptive, difficulty-aware reasoning. Our code is open-sourced at https://github.com/AutoAgents-ai/DAAO
Graph self-supervised learning (GSSL) alleviates the graph data labeling bottleneck without supervision, enabling wide application in domains like recommendation systems and social network analysis. High-frequency signals are valuable in GSSL for capturing local structural preferences, thereby enriching graph representations and boosting model performance. However, in practical applications, two critical problems hinder the efficient and robust use of these signals. First, the locality of high-frequency signals limits their full utilization by the model. Second, over-reliance on specific high-frequency signals will affect the model's generalization. To address the above problems, we propose the Frequency-Corrupt Based Graph Self-Supervised Learning (FC-GSSL) algorithm. Specifically, we generate corrupted graphs biased toward high-frequency signals by corrupting nodes and edges according to their low-frequency contributions. These corrupted graphs are fed as input to an autoencoder, with low-frequency and general features serving as the supervision. This compels the model to effectively fuse high- and low-frequency signals, thereby integrating and utilizing more valuable high-frequency information. Additionally, we design multiple sampling strategies and form diverse corrupted graphs based on the intersections and union between the results obtained from these strategies. By aligning the node representations from these views, the model can identify valuable frequency combinations, which helps reduce the negative impact of specific high-frequency components and improve generalization. FC-GSSL optimizes the design of GSSL for web applications, significantly improving model performance on complex web-related graphs, such as social networks and citation networks. This work makes a direct contribution to advancing the ''Graph Algorithms and Modeling for the Web'' research track. Experimental results on 14 datasets across multiple tasks illustrate the superiority of the proposed approach.
The increasing frequency and severity of climate-related disasters have intensified the need for real-time monitoring, early warning, and informed decision-making. Earth Observation (EO), powered by satellite data and Machine Learning (ML), offers powerful tools to meet these challenges. Foundation Models (FMs) have revolutionized EO ML by enabling general-purpose pretraining on large-scale remote sensing datasets. However most existing models rely on high-resolution satellite imagery with low revisit rates, limiting their suitability for fast-evolving phenomena and time-critical emergency response. In this work, we present HighFM, a first cut approach towards a FM for high-temporal-resolution, multispectral EO data. Leveraging over 2 TB of SEVIRI imagery from the Meteosat Second Generation (MSG) platform, we adapt the SatMAE masked autoencoding framework to learn robust spatiotemporal representations. To support real-time monitoring, we enhance the original architecture with fine-grained temporal encodings to capture short-term variability. The pretrained models are then fine-tuned on cloud masking and active fire detection tasks. We benchmark our SEVIRI-pretrained Vision Transformers against traditional baselines and recent geospatial FMs, demonstrating consistent gains across both balanced accuracy and IoU metrics. Our results highlight the potential of temporally dense geostationary data for real-time EO, offering a scalable path toward foundation models for disaster detection and tracking.
CT-to-PET synthesis aims to synthesize PET images from the widely available and lower-cost CT scans to address the high cost and additional radiation exposure associated with PET scanning. However, CT-to-PET synthesis faces two key challenges due to the sequential correlation of volumetric imaging: preserving smooth transition between adjacent slices and ensuring style consistency across long-range slices. To overcome these limitations, we propose the Style-aware Bidirectional Stream Diffusion Model, which ensures both inter-slice continuity and global style consistency with low computational cost. Specifically, we first utilize a Vector Quantized Variational Autoencoder to encode bidirectional adjacent CT slices into latent codes. A Neighbor Attention module is then introduced to capture transition patterns among these latent codes, ensuring structural continuity. To enhance style consistency, we further design a Prototype Prompting mechanism to construct a feature pool from long-range slices. Global style prototypes are extracted from the feature pool and dynamically integrated into the generation process, guiding the model’s attention toward consistent stylistic features across slices. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods.
Social bots threaten online platforms by spreading disinformation and manipulating public discourse. Graph neural networks have emerged as effective tools for bot detection by modeling user interactions, yet two fundamental challenges limit their practical deployment: severe class imbalance where bots constitute a small minority of users, and camouflaged edges where bots forge deceptive connections to humans to evade detection. Class imbalance causes decision boundaries to shift toward the majority class, while camouflaged edges corrupt neighborhood aggregation through spurious message passing. We present BotVA, a unified framework addressing both challenges through variational feature augmentation and adversarial graph learning. Our approach makes three key contributions: (i) a conditional variational autoencoder that models minority class distributions and synthesizes semantically coherent features, effectively expanding minority support in representation space; (ii) an adversarial training paradigm where a generator simulates camouflage by injecting deceptive edges while a graph transformer discriminator with semantic attention learns to identify and downweight such perturbations; and (iii) a two-stage training strategy that pretrains the variational module before alternating generator-discriminator optimization to ensure stable convergence. Experiments on three benchmarks demonstrate that BotVA achieves state-of-the-art accuracy and F1-score, exhibits strong robustness under camouflage perturbations, and maintains competitive performance with limited supervision.
Large Language Models (LLMs) powerful generative capabilities also pose significant risks, underscoring the need for effective detoxification methods to ensure safer deployment. Due to the polysemantic nature of LLM neurons, recent neuron intervention methods inevitably entangle unrelated concepts, compromising generation quality and interpretability. Sparse Autoencoders (SAEs) have opened new horizons for decomposing model activations into monosemantic features, offering interpretability and targeted feature-level steering. Empirical findings reveal that, despite capturing interpretable features, indiscriminate interventions on toxicity-related features expose the fragility of LLMs, achieving toxicity mitigation at the cost of degraded fluency. Building upon this finding, we propose DeLFE, a lightweight controlled detoxification approach that identifies specific toxic features across model layers and performs targeted interventions on them. DeLFE learns toxicity subspaces from label-guided SAE feature subsets to characterize toxic v.s. non-toxic activation patterns. When auto-completing a response token-by-token, DeLFE tracks the toxicity-triggering risks and steers toxic features away from the subspace via a flow-matching feature transformation. We further design three feature-level strategies that adjust intervention timing and strength to reconstruct the target model’s original activations. Extensive experiments demonstrate that our method achieves strong detoxification effectiveness while maintaining high generation quality across models of varying sizes and diverse base LLMs.
Graph Masked AutoEncoders (GMAEs) have achieved notable success in diverse downstream tasks. However, their superior performance usually relies on reliable and unperturbed input graphs. In real-world scenarios, graphs are vulnerable to adversarial attacks (e.g., perturbations on node features or topology). As a result, existing GMAEs tend to overfit these perturbed inputs, leading to poor robustness. Motivated by this, we propose an adversarial masked graph modeling for robust graph autoencoders, named ArmorGAE. Specifically, we first design a dual-consistency graph augmentation method that enriches the graph topology to mitigate structural vulnerabilities. Theoretically, we establish a connection between ArmorGAE and contrastive learning. It demonstrates that our augmentation strategy in GMAEs is equivalent to effectively expanding the set of high-quality positive samples in contrastive learning, providing richer and more reliable supervisory signals. We then present an adversarial masked autoencoder to mitigate model overfitting to the perturbed graph. The generator and discriminator are jointly trained under an adversarial paradigm, where the former aims to recover the underlying true data distribution, while the latter distinguishes reconstructed and real edges. Experimental results on five datasets demonstrate that ArmorGAE outperforms state-of-the-art methods in terms of classification accuracy on clean graphs and adversarial robustness under 16 distinct adversarial scenarios. The codes are publicly available at https://github.com/ZZY-GraphMiningLab/ArmorGAE.
Multi-view clustering has garnered significant attention for its ability to integrate heterogeneous data and uncover underlying categorical structures. However, prevailing autoencoder-based methods often yield indiscriminative embeddings, rendering them prone to trivial solutions. While diffusion models have shown promise in modeling complex data distributions, their generative processes remain inherently unstable. More critically, the resulting latent representations often lack sufficient intra-class compactness and inter-class separability, leading to suboptimal clustering performance. In this paper, we propose SCD-MVC, a Stable Conditional Diffusion-driven framework for Multi-View Clustering. Specifically, it integrates conditional diffusion generation into the latent encoder to generate a target view conditioned on the remaining views, thereby facilitating robust joint distribution modeling. To align the generative process with clustering objectives, we introduce a clustering-guided semantic alignment module. This module regulates the diffusion path to enforce the learning of representations that are both view-consistent and discriminative. Furthermore, we introduce a temporal consistency regularization that enforces alignment between representations of adjacent diffusion steps. This stabilizes training and guides the optimization toward clustering-friendly representations. Extensive experiments on nine benchmark datasets demonstrate that SCD-MVC consistently outperforms eight state-of-the-art methods, achieving performance gains ranging from 6.68% to 13.09%. The source code is available at https://github.com/Knighttt0011/SCD-MVC.
Generative models continue to advance rapidly in both fidelity and diversity, posing increasing challenges for reliably distinguishing generated images from real ones. Existing reconstruction-based detection methods often rely on assumptions tied to specific generative models, which leads to limited cross-model generalization and substantial computational overhead. In this paper, we propose General Feature Reconstruction Error (GFRE), a fast and generalizable detection paradigm that leverages reconstruction behavior in a general-purpose representation space. Our key insight is that real and generated images exhibit consistently different reconstruction stability when projected into universal visual representations. Instead of tracing generator-specific artifacts, GFRE employs a lightweight autoencoder to model the reconstructability of image representations, producing a reconstruction signal that is inherently generator-agnostic and transferable across diverse generative processes. Extensive experiments on images synthesized by 18 different generative models demonstrate that GFRE consistently outperforms existing state-of-the-art methods, achieving a 4.70% improvement in detection accuracy and an 8.20% gain in cross-model generalization. Moreover, by avoiding costly generator inversion and diffusion-based reconstruction, GFRE reduces reconstruction error extraction time by up to 150x, enabling efficient and scalable deployment.
Lithology-Aware Conditional Variational Autoencoder for Synthetic Well Log Generation in Petroleum Reservoirs (Student Abstract)
PDF ↗Machine learning applications in reservoir modeling are hindered by the limited availability of well log data, a common challenge in the oil and gas industry. We propose VAEc-tMC, a domain-informed Conditional Variational Autoencoder that generates synthetic well-log data conditioned on rock type. Addressing a critical gap by existing generative models that rely solely on statistical reconstruction, our model embeds geological domain knowledge into the latent space, and optimizes a modified objective with an adaptive Student-t reconstruction loss and a beta-weighted KL regularizer, improving stability under heavy-tailed data. When used for data augmentation, the synthetic samples preserve inter-log dependencies and substantially enhance downstream classification, accuracy 39→63%, F1-score 36→68%, AUC 0.46→0.80 on a held-out well. Beyond the geological context, the proposed approach illustrates a generalizable strategy where domain-aware generative models with adaptive loss functions provide a robust solution for data-efficient learning in scientific domains facing data scarcity, noise, and heavy-tailed distributions.
Tabular data synthesis presents unique challenges, with Transformer models remaining underexplored despite the applications of Variational Autoencoders and Generative Adversarial Networks. To address this gap, we propose the Transformer-based Tabular Variational AutoEncoder (TTVAE), leveraging the attention mechanism for capturing complex data distributions. The inclusion of the attention mechanism enables our model to understand complex relationships among heterogeneous features, a task often difficult for traditional methods. TTVAE facilitates the integration of interpolation within the latent space during the data generation process. Specifically, TTVAE is trained once, establishing a low-dimensional representation of real data, and then various latent interpolation methods can efficiently generate synthetic latent points. Through extensive experiments on diverse datasets, TTVAE consistently achieves state-of-the-art performance, highlighting its adaptability across different feature types and data sizes. This innovative approach, empowered by the attention mechanism and the integration of interpolation, addresses the complex challenges of tabular data synthesis, establishing TTVAE as a powerful solution.
Feature Integration Spaces: Joint Training Reveals Dual Encoding in Neural Network Representations
PDF ↗Current sparse autoencoder (SAE) approaches to neural network interpretability assume that activations can be decomposed through linear superposition into sparse, interpret-able features. Despite high reconstruction fidelity, SAEs consistently fail to eliminate polysemanticity and exhibit pathological behavioral errors. We propose that neural networks encode information in two complementary spaces com-pressed into the same substrate: feature identity and feature integration. To test this dual encoding hypothesis, we develop sequential and joint-training architectures to capture identity and integration patterns simultaneously. Joint training achieves 41.3% reconstruction improvement and 51.6% reduction in KL divergence errors. This architecture spontaneously develops bimodal feature organization: low squared norm features contributing to integration pathways and the rest contributing directly to the residual. Small nonlinear components (3% of parameters) achieve 16.5% standalone improvements, demonstrating parameter-efficient capture of computational relationships crucial for behavior. Additionally, intervention experiments using 2×2 factorial stimulus designs demonstrated that integration features exhibit selective sensitivity to experimental manipulations and produce systematic behavioral effects on model outputs, including significant interaction effects across semantic dimensions. This work provides systematic evidence for (1) dual encoding in neural representations, (2) meaningful nonlinearly encoded feature interactions, and (3) introduces an architectural paradigm shift from post-hoc feature analysis to integrated computational design, establishing foundations for next-generation SAEs.
EA-VAE: Learning to Reconstruct Dysarthric Speech via Variational Autoencoder with Encoding Alignment
PDF ↗Dysarthric speech reconstruction (DSR) aims to enhance the intelligibility of dysarthric speech. Compared with normal speech, the dysarthric speech is characterized by its pathological features, including discontinuous pronunciation, slow speech, hoarseness, and improper pauses. Significant disparities in the feature space between normal and dysarthric speech may result in suboptimal speech reconstruction, thereby degrading speech intelligibility. To enhance the reconstruction ability of speech feature spaces, this paper proposes a DSR model named the Encoding-Aligned Variational Autoencoder (EA-VAE). By incorporating alignment modules of frame-level embedding features, prior distributions, and duration into the encoder of the VAE, the model explicitly aligns the dysarthric speech encoding with a representation of the parallel normal speech. A shared decoder is then used to generate speech with improved intelligibility. Experimental results on the UASpeech benchmark confirm that EA-VAE achieves state-of-the-art performance, with a 31.7% relative word error rate reduction and the highest subjective MOS score (4.48), thoroughly validating the effectiveness and advancements of the proposed method in dysarthric speech reconstruction.
I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
PDF ↗Recent LLMs like DeepSeek-R1 have demonstrated state-of-the-art performance by integrating deep thinking and complex reasoning during generation. However, the internal mechanisms behind these reasoning processes remain unexplored. We observe reasoning LLMs consistently use vocabulary associated with human reasoning processes. We hypothesize these words correspond to specific reasoning moments within the models' internal mechanisms. To test this hypothesis, we employ Sparse Autoencoders (SAEs), a technique for sparse decomposition of neural network activations into human-interpretable features. We introduce ReasonScore, an automatic metric to identify active SAE features during these reasoning moments. We perform manual and automatic interpretation of the features detected by our metric, and find those with activation patterns matching uncertainty, exploratory thinking, and reflection. Through steering experiments, we demonstrate that amplifying these features increases performance on reasoning-intensive benchmarks (+2.2%) while producing longer reasoning traces (+20.5%). Using the model diffing technique, we provide evidence that these features are present only in models with reasoning capabilities. Our work provides the first step towards a mechanistic understanding of reasoning in LLMs.