Automatic echocardiography video segmentation is a powerful tool for improving the accuracy of cardiovascular function assessment. However, it remains a challenging task owing to (1) extensive speckle noise and blurred boundaries, (2) dramatic shape variations of targeting structures across frames, and (3) limited labeled data due to the high cost of annotation. In this paper, we present a novel semi-supervised segmentation model based on Vision Mamba (Vim) to comprehensively tackle these challenges; we call it EchoVim. Our framework introduces three technical innovations: First, a bidirectional inference mechanism (BIM) which can propagate label information bidirectionally from end-diastolic (ED) and end-systolic (ES) frames to generate pseudo-labels, coupled with confidence-aware dynamic updating to progressively refine supervision signals. Second, a dynamic interaction temporal alignment (DITA) module that establishes anatomical correspondence across frames by adaptively enhancing features near temporally stable regions while suppressing motion-irrelevant artifacts, effectively addressing variations in cardiac shape. Third, a semantic token-attentive refinement (STR) module that constructs low-rank semantic tokens to encode cardiac structure priors, utilizing attention-guided nonlinear transformations to disentangle speckle noise from true anatomical patterns. We conduct extensive experiments on two benchmarking echocardiography video datasets: CAMUS and EchoNet-Dynamic, and the results demonstrate that our method outperforms existing state-of-the-art approaches with real-time inference. Codes are available at https://github.com/guojx2255/EchoVim.
论文检索
输入标题、作者或关键词，从 960 篇学术成果中精准定位
The domain gap between pretraining data (e.g., ImageNet, LUPerson) and downstream ReID datasets often leads to suboptimal performance when directly fine-tuning pretrained models. While existing methods attempt to bridge this gap by incorporating additional modalities (e.g., text, 3D data) or visual cues (e.g., pose, body masks), these approaches introduce two key limitations: (1) they may distract the model with irrelevant factors like background clutter or clothing variations, and (2) they inevitably increase computational overhead during inference. To address these issues, we propose the Weak Saliency Feedback Transformer (WSFFormer), inspired by the feedback mechanisms in biological visual systems. Unlike traditional one-way feature propagation, WSFFormer employs an adaptive feedback loop during training to enhance low-response regions, enabling the model to capture richer and more discriminative features. The WSFFormer introduces three key components: (1) The Lateral Feedback Module (LFM) mimics retinal lateral inhibition by adaptively suppressing high-response regions and amplifying weak discriminative features, forcing attention on subtle details; (2) The Progressive Feedback Module (PFM) refines feedback through deep-to-shallow closed-loop propagation, blending high-level semantics with spatial details; (3) The Feedback Sensitive Entropy Loss (FSE Loss) optimizes target-domain adaptation by quantifying divergence between forward and feedback-corrected features. Experiments on holistic/occluded ReID benchmarks show WSFFormer outperforms ViT/Swin-based SOTA methods without extra inference cost.
Although object counting based on two-dimensional (2D) RGB images offers an effective solution in certain scenarios, it is significantly challenged by complex environments characterized by background noise, occlusion, depth variations, illumination changes, and other factors, often resulting in miscounting or missed detections. This issue is particularly pronounced in applications such as agriculture or retail, where fruits or products are frequently stacked in layers on shelves, severely compromising counting accuracy. To address these limitations, we propose TrueCount, a novel method that integrates segmented images, depth information, and other multi-modal data to enhance counting accuracy and robustness of pretrained large vision-language models (VLMs). TrueCount introduces a flexible framework capable of simultaneously processing multiple modal signals, including 2D RGB images, segmentation, and depth maps, while supporting both textual and visual prompts. During training and inference, TrueCount performs cross-attention and self-attention across all inputs and prompts. These features are then decoded to localize prompt features within the input, thereby jointly optimizing the model's counting capability. Additionally, TrueCount dynamically assesses the confidence of each modality for accurate counting in a given context, enabling effective fusion and complementary utilization of multi-modal information. Extensive experiments on multiple benchmarks including FSC-147 and CountBench demonstrate that TrueCount surpasses the previous state-of-the-art, e.g. achieving a new minimum mean average error of 4.64 on FSC-147.
WebAssembly (Wasm) is a platform-independent, low-level binary language that enables near-native performance in web applications. Given its growing importance in the web ecosystem, securing WebAssembly programs becomes increasingly important. A key security concern with WebAssembly is the threat of instruction-timing side-channel attacks, which exploit timing variations in branch instructions dependent on sensitive data, allowing attackers to infer sensitive information through timing measurement. In this paper, we introduce WaSCR, an automated WebAssembly instruction-timing Side-Channel Repairer. WaSCR uses control and data dependencies to trace the flow of sensitive data and prevent its leakage. It employs rule-based code transformations to linearize the program, eliminating branches dependent on sensitive data and substituting them with constant-time selectors. Our evaluation demonstrates that WaSCR effectively eliminates instruction-timing side channels while maintaining program correctness, with efficient repairs and moderate performance overhead.
Multi-modal time series data is common in web technologies like the Internet of Things (IoT). Existing methods for multi-modal time series representation learning aim to disentangle the modality-shared and modality-specific latent variables. Although achieving notable performances on downstream tasks, they usually assume an orthogonal latent space. However, the modality-specific and modality-shared latent variables might be dependent on real-world scenarios. Therefore, we propose a general generation process, where the modality-shared and modality-specific latent variables are dependent, and further develop a Multi-modAl TEmporal Disentanglement (MATE) model. Specifically, our MATE model is built on a temporally variational inference architecture with the modality-shared and modality-specific prior networks for the disentanglement of latent variables. Furthermore, we establish identifiability results to show that the extracted representation is disentangled. More specifically, we first achieve the subspace identifiability for modality-shared and modality-specific latent variables by leveraging the pairing of multi-modal data. Then we establish the component-wise identifiability of modality-specific latent variables by employing sufficient changes of historical latent variables. Extensive experimental studies on 12 datasets show a general improvement in different downstream tasks, highlighting the effectiveness of our method in real-world scenarios.
Online continual learning (CL) is becoming a mainstream paradigm to learn incrementally from task streams without forgetting previously learned knowledge. However, the current online CL primarily focuses on learning performance, such as avoiding catastrophic forgetting, neglecting the critical demands of system performance, such as real-time inference. As a result, the performance of real-time inference in online CL degrades significantly due to frequent data distribution variations and time-consuming model adaptation. In this work, we propose ELITE, an online CL framework with device-cloud collaboration, to realize on-device real-time inference on time-varying task streams with performance guarantee. To realize on-device real-time inference in online CL, ELITE features a new design of the model zoo comprising various pre-trained models with the assistance of the cloud, and proposes a task-oriented on-device model selection to quickly retrieve the best-fit models instead of performing time-consuming model retraining. To prevent performance degradation on new tasks not available in the cloud, we introduces a latency-aware on-device model fine-tuning strategy to adapt to new tasks with an accuracy-latency trade-off, and dynamically updates the model zoo to enhance ELITE. Extensive evaluations on five real-world datasets have been conducted, and the results demonstrate that ELITE consistently outperforms the state-of-art solutions, improving the accuracy by 16.3% on average and reducing the response latency by up to 1.98 times.
Exploring Hypergraph Condensation via Variational Hyperedge Generation and Multi-Aspectual Amelioration
PDF ↗Hypergraph neural networks (HyperGNNs) show promise in modeling online networks with high-order correlations. Despite notable progress, training these models on large-scale raw hypergraphs entails substantial computational and storage costs, thereby increasing the need of hypergraph size reduction. However, existing size reduction methods primarily capture pairwise association pattern within conventional graphs, making them challenging to adapt to hypergraphs with high-order correlations. To fill this gap, we introduce a novel hypergraph condensation framework, HG-Cond, designed to distill large-scale hypergraphs into compact, synthetic versions while maintaining comparable HyperGNN performance. Within this framework, we develop a Neural Hyperedge Linker to capture the high-order connectivity pattern through variational inference, achieving linear complexity with respect to the number of nodes. Moreover, We propose a multi-aspectual amelioration strategy including a Gradient-Parameter Synergistic Matching objective to holistically refine synthetic hypergraphs by coordinating improvements in node attributes, high-order connectivity, and label distributions. Extensive experiments demonstrate the efficacy of HG-Cond in hypergraph condensation, notably outperforming the original test accuracy on the 20News dataset while concurrently reducing the hypergraph size to a mere 5% of its initial scale. Furthermore, the condensed hypergraphs demonstrate robust cross-architectural generalizability and potential for expediting neural architecture search.
Climate activism is crucial in stimulating collective societal and behavioral change towards sustainable practices through political pressure. Although multiple factors contribute to the participation in activism, their complex relationships and the scarcity of data on their interactions have restricted most prior research to studying them in isolation, thus preventing the development of a quantitative, causal understanding of why people approach activism. In this work, we develop a comprehensive causal model of how and why Reddit users engage with activist communities driving mass climate protests (mainly the 2019 Earth Strike, Fridays for Future, and Extinction Rebellion). Our framework, based on Stochastic Variational Inference applied to Bayesian Networks, learns the causal pathways over multiple time periods. Distinct from previous studies, our approach uses large-scale and fine-grained longitudinal data (2016 to 2022) to jointly model the roles of sociodemographic makeup, experience of extreme weather events, exposure to climate-related news, and social influence through online interactions. We find that among users interested in climate change, participation in online activist communities is indeed influenced by direct interactions with activists and largely by recent exposure to media coverage of climate protests. Among people aware of climate change, left-leaning people from lower socioeconomic backgrounds are particularly represented in online activist groups. Our findings offer empirical validation for theories of media influence and critical mass, and lay the foundations to inform interventions and future studies to foster public participation in collective action.
Graph Neural Networks (GNNs)-based recommendation algorithms typically assume that training and testing data are drawn from independent and identically distributed (IID) spaces. However, this assumption often fails in the presence of out-of-distribution (OOD) data, resulting in significant performance degradation. In this study, we construct a Structural Causal Model (SCM) to analyze interaction data, revealing that environmental confounders (e.g., the COVID-19 pandemic) lead to unstable correlations in GNN-based models, thus impairing their generalization to OOD data. To address this issue, we propose a novel approach, graph representation learning via causal diffusion (CausalDiffRec) for OOD recommendation. This method enhances the model's generalization on OOD data by eliminating environmental confounding factors and learning invariant graph representations. Specifically, we use backdoor adjustment and variational inference to infer the real environmental distribution, thereby eliminating the impact of environmental confounders. This inferred distribution is then used as prior knowledge to guide the representation learning in the reverse phase of the diffusion process to learn the invariant representation. In addition,we provide a theoretical derivation that proves optimizing the objective function of CausalDiffRec can encourage the model to learn environment-invariant graph representations, thereby achieving excellent generalization performance in recommendations under distribution shifts. Our extensive experiments validate the effectiveness of CausalDiffRec in improving the generalization of OOD data, and the average improvement is up to 10.69% on Food, 18.83% on KuaiRec, 22.41% on Yelp2018, and 11.65% on Douban datasets.
With the rise of web-based technologies, online tutoring platforms have emerged to provide personalized learning services by modeling learners' engagement behaviors, improving both convenience and efficiency in academic progress. Cognitive diagnosis has been always recognized a essential learner modeling task in personalized education, which aims to infer learners' mastery in specific knowledge concepts by mining and analyzing their practice behavior. However, most existing studies fail to explicitly disentangling the multiple interdependent factors that influence learner's response feedback during the problem-solving process, both in web-based environments and real-world contexts. To address this issue, we propose DISCD, a feedback-centric DIS entangled Cognitive Diagnosis framework for enhancing effective and interpretable learner modeling. Specifically, we first introduce a feedback-centric disentangled encoder grounded in variational inference to effectively characterize learners' cognitive traits by modeling their practice responses. To achieve this, we fully leverage the interaction matrix and the exercise-concept correlation matrix to extract implicit signals in the disentanglement process, employing three dedicated sub-encoders to efficiently and comprehensively capture these attributes. Next, we develop a multi-level cognitive coordination module to systematically model the disentangled cognitive factors, ensuring their seamless integration into the diagnosis decoding process. Finally, we design a cognitive interaction decoder to reconstruct and refine learners' engagement trajectories in exercises. Extensive experiments on four educational datasets validate the effectiveness of the proposed DISCD model in learner modeling for cognitive diagnosis.
Interference in online controlled experiments poses significant challenges, as treatment effects may propagate through user interactions. This phenomenon systematically biases conventional treatment effect estimators and substantially complicates the interpretation of experimental results. Detecting the presence of interference is crucial for ensuring the reliability of conclusions, ultimately enhancing decision-making processes. We formalize a framework for detecting the existence of interference with dyadic data, which is ubiquitous in social network platforms. Dyadic data, typically generated from user interactions, have not been fully utilized in online controlled experiments. We first introduce the dyadic-robust inference techniques, then illustrate how the dyadic outcomes link to the user-level metrics and how interference manifests. We then propose a comprehensive approach to detect interference effects based on dyadic-robust t-statistics, and we also provide a subsampling approach to reduce the computational complexity of the proposed methods. We also provide several approaches to leverage pre-experiment data, such as selecting the optimal treatment probability, designing stratified randomization in the pre-experiment stage, or reducing the variance with both user-level and dyad-level control variates at the post-experiment stage. Our approach is computationally efficient and easily integrated with existing pipelines. We provide theoretical guarantees for our inference procedure across various asymptotic regimes. Finally, we conduct several empirical evaluations under various dyadic data-generating processes to assess the performance and robustness of the proposed procedures. Finally, we introduce an interference detection pipeline and demonstrate it with an experiment from Weixin.
A fundamental problem in many science domains is learning the causal structure of a system from observed data. The observed data canonically come from multiple environments (i.e. different times, locations, and measurements), and causal models may have unobserved shifts. Although the causal graphs can be identified by modeling the distribution changes among different environments, existing solutions can only learn causal structures when given environmental information. In contrast, we propose a causal discovery approach (CausalSME) which automatically identifies pseudo environments and unobserved distribution shifts. Specifically, CausalSME learns a causal model containing unobserved variables, which can correct the distribution shifts with mixed environments. The heart of CausalSME is a variational autoencoder that infers shifted causal effects of unobserved variables and guides the identification of environment information. It further divides the shifted samples by the identified environments to jointly learn an invariant causal model. We prove the structure identifiability of CausalSME with the causal additive model. In our extensive experiments we show that CausalSME achieves state-of-the-art performance.
This work addresses a key limitation in current federated learning approaches, which predominantly focus on homogeneous tasks, neglecting the task diversity on local devices. We propose a principled integration of multi-task learning using multi-output Gaussian processes (MOGP) at the local level and federated learning at the global level. MOGP handles correlated classification and regression tasks, offering a Bayesian non-parametric approach that naturally quantifies uncertainty. The central server aggregates the posteriors from local devices, updating a global MOGP prior redistributed for training local models until convergence. Challenges in performing posterior inference on local devices are addressed through the Polya-Gamma augmentation technique and mean-field variational inference, enhancing computational efficiency and convergence rate. Experimental results on both synthetic and real data demonstrate superior predictive performance, OOD detection, uncertainty calibration and convergence rate, highlighting the method's potential in diverse applications. Our code is publicly available at https://github.com/JunliangLv/task_diversity_BFL.
Clustering high-dimensional directional data (i.e., L2 normalized vectors) presents significant challenges due to the intricate spherical representations of latent embeddings and the limitations of classical (non-deep) clustering techniques. Moreover, dynamically inferring the number of clusters remains a fundamental issue in existing deep clustering methods, especially those involving complex model-selection criteria. This paper addresses these challenges by introducing a novel deep nonparametric clustering framework that employs hyperspherical latent embeddings within a Variational Autoencoder architecture, enhanced by an infinite Von Mises-Fisher Mixture Model as a dynamic prior. This approach enables automatic adaptation of cluster numbers during training, eliminating the need for predefined clusters and traditional model selection processes. Our scalable architecture effectively integrates In-vMFMM with hyperspherical embeddings to tackle the complexities of directional data. Utilizing a joint training strategy, our method alternates between updating neural network parameters and adjusting mixture model priors via nonparametric variational Bayes. Empirical evaluations on benchmark datasets, including complex ImageNet-50, demonstrate that our approach significantly outperforms state-of-the-art deep nonparametric clustering methods. It also robustly estimates the number of clusters, showcasing its effectiveness and versatility in handling high-dimensional directional data.
Test-time adaptation aims to adapt pre-trained deep neural networks using solely online unlabelled test data during inference. Although TTA has shown promise in visual applications, its potential in time series contexts remains largely unexplored. Existing TTA methods, originally designed for visual tasks, may not effectively handle the complex temporal dynamics of real-world time series data, resulting in suboptimal adaptation performance. To address this gap, we propose Augmented Contrastive Clustering with Uncertainty-aware Prototyping (ACCUP), a straightforward yet effective TTA method for time series data. Initially, our approach employs augmentation ensemble on the time series data to capture diverse temporal information and variations, incorporating uncertainty-aware prototypes to distill essential characteristics. Additionally, we introduce an entropy comparison scheme to selectively acquire more confident predictions, enhancing the reliability of pseudo labels. Furthermore, we utilize augmented contrastive clustering to enhance feature discriminability and mitigate error accumulation from noisy pseudo labels, promoting cohesive clustering within the same class while facilitating clear separation between different classes. Extensive experiments conducted on three real-world time series datasets demonstrate the effectiveness and generalization potential of the proposed method, advancing the underexplored realm of TTA for time series data. Our code is available at https://github.com/Tokenmw/ACCUP-main.
Phenotypic Profile-Informed Generation of Drug-Like Molecules via Dual-Channel Variational Autoencoders
PDF ↗The de novo generation of drug-like molecules capable of inducing desirable phenotypic changes is receiving increasing attention. However, previous methods predominantly rely on expression profiles to guide molecule generation, but overlook the perturbative effect of the molecules on cellular contexts. To overcome this limitation, we propose SmilesGEN, a novel generative model based on variational autoencoder (VAE) architecture to generate molecules with potential therapeutic effects. SmilesGEN integrates a pre-trained drug VAE (SmilesNet) with an expression profile VAE (ProfileNet), jointly modeling the interplay between drug perturbations and transcriptional responses in a common latent space. Specifically, ProfileNet is imposed to reconstruct pre-treatment expression profiles when eliminating drug-induced perturbations in the latent space, while SmilesNet is informed by desired expression profiles to generate drug-like molecules. Our empirical experiments demonstrate that SmilesGEN outperforms current state-of-the-art models in generating molecules with higher degree of validity, uniqueness, novelty, as well as higher Tanimoto similarity to known ligands targeting the relevant proteins. Moreover, we evaluate SmilesGEN for scaffold-based molecule optimization and generation of therapeutic agents, and confirmed its superior performance in generating molecules with higher similarity to approved drugs. SmilesGEN establishes a robust framework that leverages gene signatures to generate drug-like molecules that hold promising potential to induce desirable cellular phenotypic changes. The source code and datasets are available at: https://github.com/hliulab/SmilesGEN.
Proactive scheduling creates robust offline schedules that optimize resource utilization and minimize job flow times. This work addresses scheduling challenges in business processes, often encountered in service systems, which differ from traditional applications like manufacturing due to inherent uncertainties in activity durations, and human resource availability. We model the business process scheduling problem (BPSP) as a variation of stochastic resource-constrained multi-project scheduling (RCMPSP), and apply process mining to infer unknown parameter values from historical event data. To overcome the randomness in activity durations, we transform the problem into its deterministic counterpart, and prove that the latter provides a lower bound on the Makespan of the stochastic problem. Our approach integrates data-driven Monte Carlo simulation with constraint programming to generate proactive schedules that guarantee, with high probability, that the Makespan remains below a predefined threshold. We evaluate our approach using synthetic datasets with varying levels of uncertainty and size. In addition, we apply the approach to a real-world dataset from an outpatient cancer hospital, demonstrating its effectiveness in optimizing the process Makespan by an average of 5% to 14%.
Machine learning techniques have gained considerable attention in the field of empirical asset pricing. Conditioning on a broad set of firm characteristics, one of the most popular no-arbitrage workhorses is a nonlinear conditional asset pricing model that consists of two modules within a neural network structure, i.e., factor and beta estimates, for which we propose a novel contrastive adversarial variational Bayes (CAVB) framework. To exploit the factor structure, we employ adversarial variational Bayes that transforms the maximum-likelihood problem into a zero-sum game between a variational autoencoder (VAE) and a generative adversarial network (GAN), where an auxiliary discriminative network brings in arbitrary expressiveness to the inference model. To tackle the problem of learning indistinguishable feature representations in the beta network, we introduce a contrastive loss to learn distinctive hidden features of the factor loadings in correspondence to conditional quantiles of return distributions. CAVB establishes a robust relation between the cross-section of asset returns and the common latent factors with nonlinear factor loadings. Extensive experiments show that CAVB not only significantly outperforms prominent models in the existing literature in terms of total and predictive R-squares, but also delivers superior Sharpe ratios after transaction costs for both long-only and long-short portfolios.
QuantileFormer: Probabilistic Time Series Forecasting with a Pattern-Mixture Decomposed VAE Transformer
PDF ↗Probabilistic time series forecasting has attracted an increasing attention in machine learning community for its potential applications in the fields of renewable energy, traffic management, healthcare, etc. Previous research mainly focused on extracting long-range dependencies for point-wise prediction, which fail to capture complex temporal patterns and statistical characteristics for probabilistic analysis. In this paper, we propose a novel pattern-mixture decomposition method that decomposes long-term series into quantile drift, divergence patterns, and Gaussian mixture components, which can effectively capture the intricate temporal patterns and stochastic characteristics in time series. Based on pattern-mixture decomposition, we propose a novel Transformer-based model called QuantileFormer for probabilistic time series forecasting. It takes the the comprehensive drift-divergence mixture patterns as features, and designs a variational inference based fusion Transformer architecture to generate quantile prediction results. Extensive experiments show that the proposed method consistently boosts the baseline methods by a large margin and achieves state-of-the-art performance on six real-world benchmarks.
Low-rank Adaptation (LoRA) has demonstrated remarkable capabilities for task specific fine-tuning. However, in scenarios that involve multiple tasks, training a separate LoRA model for each one results in considerable inefficiency in terms of storage and inference. Moreover, existing parameter generation methods fail to capture the correlations among these tasks, making multi-task LoRA parameter generation challenging. To address these limitations, we propose In-Context Meta LoRA (ICM-LoRA), a novel approach that efficiently achieves task-specific customization of large language models (LLMs). Specifically, we use training data from all tasks to train a tailored generator, Conditional Variational Autoencoder (CVAE). CVAE takes task descriptions as inputs and produces task-aware LoRA weights as outputs. These LoRA weights are then merged with LLMs to create task-specialized models without the need for additional fine-tuning. Furthermore, we utilize in-context meta-learning for knowledge enhancement and task mapping, to capture the relationship between tasks and parameter distributions. As a result, our method achieves more accurate LoRA parameter generation for diverse tasks using CVAE. ICM-LoRA enables more accurate LoRA parameter reconstruction than current parameter reconstruction methods and is useful for implementing task-specific enhancements of LoRA parameters. At the same time, our method occupies 283MB, only 1% storage compared with the original LoRA. The code is available at https://github.com/YihuaJerry/ICM-LoRA.