论文检索

General Machine Learning · Unsupervised and Semi-supervised Learning

Hong-Phuc Phan, Tuan Vu, Tung Kieu, Sơn Hà Xuân, Bin Yang, Christian S Jensen

Unsupervised outlier detection is attractive because it eliminates the need for labeled data. Further, forming multi-model ensembles can improve detection robustness performance. However, composing an ensemble without labeled data is challenging. Naively composing ensembles can cause ensemble saturation, where redundant or unreliable detection models degrade performance and incur unnecessary computations. We propose MetaEns, an automatic unsupervised framework for the selection of outlier detection model ensembles. Using labeled meta-datasets, MetaEns learns a model that predicts marginal ensemble gains that estimate the expected improvement of adding a candidate model to a partially constructed ensemble. At test time, this learned signal is combined with a submodular-inspired proxy objective that enforces diminishing returns through diversity-aware discounting and family-level risk regularization, thereby enabling greedy sequential selection with adaptive early stopping. As a result, MetaEns constructs compact, high-quality ensembles without access to ground-truth labels. Experiments on 39 real-world datasets show that MetaEns is able to consistently outperform state-of-the-art unsupervised selectors and ensemble baselines, achieving higher average precision while using fewer models.

General Machine Learning · Unsupervised and Semi-supervised Learning

Cheng Lu, Mengxin Wang, Dennis Zhang, Heng Zhang

Large language models enable inexpensive AI-generated annotations, but using them reliably for causal inference remains challenging. Naively pooling AI and human data induces bias, while existing methods such as Prediction-Powered Inference (PPI; Angelopoulos et al., 2023) treat AI outputs as proxies of true labels - an assumption often violated for generative model outputs in practice. We propose Generative Augmented Inference (GAI), a framework that treats AI outputs as general, potentially high-dimensional informative features for learning human labels rather than as surrogates. GAI flexibly models this relationship using nonparametric methods, enabling consistent estimation and valid inference from combined human and AI data. We establish asymptotic normality and show that GAI strictly improves asymptotic efficiency over human-data-only estimation whenever AI outputs are informative for true labels. Empirical studies on real-world datasets demonstrate that GAI significantly reduces estimation error and improves confidence interval quality across diverse generative data sources relative to human-only and PPI-based estimation.

General Machine Learning · Unsupervised and Semi-supervised Learning

Senyu Hou, Gaoxia Jiang, Xinyi Zheng, Yaqing Guo, Shuna Liang, Wenjian Wang

Label noise is pervasive in real-world datasets and significantly compromises model generalization, fueling extensive research into Learning with Noisy Labels (LNL). Most LNL methods focus on robust discriminative learning, while recent generative classifiers such as label diffusion models (LDMs) show superior robustness by modeling class posteriors. However, current LDMs predominantly rely on standard $\epsilon$-prediction, where Gaussian pnoise lacks explicit class semantics, limiting both optimization and inference under label noise environments. To address this issue, we propose just y-prediction (JYP), a novel training paradigm that enables LDMs to directly characterize the label manifold and leverage explicit class-semantic guidance. Theoretically, we prove that JYP converges to an optimal solution equivalent to that of $\epsilon$-prediction within the label diffusion framework, while facilitating accelerated convergence and enabling one-step inference. Leveraging JYP as a foundation, we further incorporate historical cumulative inconsistency to adaptively tailor optimization strategies for clean, noisy, and hard samples. Extensive experiments demonstrate that our method consistently outperforms competitors across diverse synthetic noisy datasets and achieves state-of-the-art performance on multiple real-world benchmarks.

General Machine Learning · Unsupervised and Semi-supervised Learning

Jiahao Shi, Omar Hagrass, Jason Klusowski

In many prediction problems, we have extra information during training (for example, measurements that are expensive or slow to collect) that will not be available when the model is deployed. A common strategy is to first train a model that uses all training information, then use its predictions on unlabeled examples to train a second model that only uses the inputs available at test time. However, when the extra training-only information is weak or noisy, this two-step approach can mislead the deployable model and even hurt accuracy. We propose a joint training method that learns the two models together, so the deployable model can benefit from the extra information only when it actually helps, instead of inheriting its mistakes. We provide guarantees that describe when joint training improves prediction accuracy and analyze a simple alternating training algorithm for large, high-dimensional models. Experiments on synthetic data and medical prediction tasks show that our approach avoids these failures and consistently outperforms standard two-step baselines.

General Machine Learning · Unsupervised and Semi-supervised Learning

Zhengzheng Lou, Yuhan Zhan, Mingyang Lv, Yingxuan Li, Yuyang Du, Shizhe Hu

Multi-modal clustering, which integrates information from diverse sources and feature modalities, has shown great potential in data mining and computer vision. However, existing methods relying on single-granularity relationships often struggle with complex data distributions, leading to limited performance, as fine-grained features are prone to local heterogeneity and redundant perturbations while coarse-grained representations tend to lose local structural information. To address these limitations, we introduce granular-balls (GBs), adaptive multi-granularity hyperspheres that enclose similar samples, and propose the Structure-aware Granular-Ball based Information Bottleneck (SGB-IB) algorithm. This method initializes the dataset as a single GB and recursively splits GBs based on a purity metric, which quantifies the average mutual information between sample features and K-means-derived pseudo-labels across all modalities. It also balances local structure preservation and global redundancy suppression through a structure-aware objective function. Extensive experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art approaches, validating the effectiveness of fusing GB structures with information-theoretic principles.

General Machine Learning · Unsupervised and Semi-supervised Learning

Xiaoyu Wang, Zhuoming Li, Bo Han, Hui LIU, Junhui Hou, Yuheng Jia

Single-Positive Multi-Label Learning (SPML) studies learning from incomplete supervision, where each instance is annotated with only one positive label despite potentially belonging to multiple categories. While existing methods assume the annotated labels are randomly distributed, real-world annotations are often biased toward the most salient category. We formalize this realistic scenario as Salient Single-Positive Multi-Label Learning (SalSPML). This salient annotation bias poses a challenge to conventional SPML methods, as the missing labels often correspond to less salient and harder-to-recognize categories. Fortunately, we find that salient annotations are typically more representative and informative. Motivated by this insight, we propose Prototype-Guided Rejection for Salient Annotation (PiSA), which constructs reliable class-wise prototypes from salient labels and leverages them to guide embedding learning for non-salient labels recognition. We theoretically demonstrate that SalSPML is harder than Random SPML due to irreducible annotation bias, and under SalSPML, more accurate prototypes facilitate false-negative label detection. Experiments on multiple benchmarks, together with two newly constructed real-world SalSPML datasets, demonstrate that PiSA consistently outperforms existing methods, achieving an average mAP improvement of 3.16\%. Our code is available in the supplementary materials.

General Machine Learning · Unsupervised and Semi-supervised Learning

Henry Li, Robin Scheibler, Efthymios Tzinis, Matt Shannon, Arnaud Doucet, john hershey

The goal of single-channel source separation is to reconstruct $K$ sources given their mixture. In supervised settings where vast amounts of clean source data are available, this challenging, ill-posed problem has been addressed successfully by generative diffusion and flow-based prior models. However, access to such clean source samples is often limited. To bridge this gap, we present Separation via Unsupervised Remixing Flow (\textbf{SURF}), an unsupervised flow matching approach for source separation that learns directly from observed mixtures. This method relies on a novel combination of state-of-the-art supervised flow matching and regression-based self-supervised techniques. At a high level, starting from a teacher model, we utilize a ``remixing'' step to bootstrap the learning of a student flow model from the teacher's estimates. We provide insights into the objectives optimized by this approach and draw a novel connection to the Wake-Sleep algorithm. Empirical evaluations on image and audio benchmarks demonstrate that \textbf{SURF} establishes a new state-of-the-art, significantly outperforming existing unsupervised methods.

General Machine Learning · Unsupervised and Semi-supervised Learning

Joonhyuk Lee, Virginia L., Sarah Zhao, Yash Nair, Asher Spector, Regev Cohen, Emmanuel J Candes

Verification of model outputs is rapidly emerging as a key primitive for both training and real-world deployment of large language models (LLMs). In practice, this often involves using imperfect LLM judges and reward models since ground truth acquisition can be time-consuming and expensive. We introduce Fully Unsupervised Score Ensembling (FUSE), a method for improving verification quality by ensembling verifiers without access to ground truth correctness labels. The key idea behind FUSE is to control conditional dependencies between verifiers in a manner that improves the unsupervised performance of a class of spectral algorithms from the ensembling literature. Despite requiring zero ground truth labels, FUSE typically matches or improves upon semi-supervised alternatives in test-time scaling experiments with diverse sets of generator models, verifiers, and benchmarks. In particular, we validate our method on both conventional academic benchmarks such as GPQA Diamond and on frontier, unsaturated benchmarks such as Humanity's Last Exam and IMO Shortlist questions.

General Machine Learning · Unsupervised and Semi-supervised Learning

Jiazhen Huang, Zhiming Liu, Changhu Wang, Wei Ju, Ziyue Qiao, Xiao Luo

Many methods aim to enhance the performance of vision-language models (VLMs) at test time. Among them, transduction has emerged as a promising paradigm due to its strong compatibility and efficiency. However, realistic evaluations often involve highly imbalanced class distributions, which cause performance degradation or even collapse. In this work, we systematically revisit transduction from the perspective of penalized likelihood estimation (PLE), showing that PLE with a KL-divergence anchor term naturally yields an adaptive shrinkage behavior between prior anchors and empirical estimates. From this viewpoint, the brittleness of transductive methods can be attributed to the absence of anchoring mechanism and static modeling of the shrinkage strength. Therefore, we propose Mixture of Von Mises-Fisher Models with Dynamic Shrinkage (MOON). MOON is based on a mixture of von Mises-Fisher distributions to model feature representations on the unit hypersphere. To handle imbalance, MOON dynamically adjusts the shrinkage strength using zero-shot priors at both instance and class levels. Thus, it suppresses unreliable assignments and prevents harmful updates from outlier classes, thereby mitigating negative transfer. MOON is model-agnostic, training-free, and requires no task-specific hyperparameter tuning. Extensive experiments further validate the advantage of MOON in both performance and efficiency.

General Machine Learning · Unsupervised and Semi-supervised Learning

Simon Roschmann, Paul KRZAKALA, Sonia Mazelet, Quentin Bouniot, Zeynep Akata

The Platonic Representation Hypothesis posits that neural networks trained on different modalities converge toward a shared statistical model of the world. Recent work exploits this convergence by aligning frozen pretrained vision and language models with lightweight alignment layers, but typically relies on contrastive losses and millions of paired samples. In this work, we ask whether meaningful alignment can be achieved with substantially less supervision. We introduce a semi-supervised setting in which pretrained unimodal encoders are aligned using a small number of image–text pairs together with large amounts of unpaired data. To address this challenge, we propose SOTAlign, a two-stage framework that first recovers a coarse shared geometry from limited paired data using a linear teacher, then refines the alignment on unpaired samples via an optimal-transport-based divergence that transfers relational structure without overconstraining the target space. Unlike existing semi-supervised methods, SOTAlign effectively leverages unpaired images and text, learning robust joint embeddings across datasets and encoder pairs, and significantly outperforming supervised and semi-supervised baselines.

General Machine Learning · Unsupervised and Semi-supervised Learning

Zelei Wu, Kun Zhou, xulun ye, Jie Hong, Jieyu Zhao, Yifan Mei

In real-world Few-Shot Learning (FSL), support sets are quickly constructed and inevitably contain noisy samples. With limited examples per class, even a single noisy instance can distort class distributions, cause prototype drift, and reduce generalization. Existing methods mostly assume clean data or require large-scale statistics, which are impractical in FSL’s data-scarce setting. We find that clean samples in semantic feature space lie in low-rank subspaces, while noisy samples cause rank anomalies disrupting this structure. To address this, we propose a differentiable low-rank approximation that estimates the intrinsic rank of the support set and detects anomalous noisy samples. Building on this, a rank-guided diffusion process generates high-quality replacements under low-rank constraints, reconstructing a clean, consistent support set for improved robustness.This low-rank guided approach effectively mitigates prototype drift and significantly reduces errors under noise levels up to 40% across MiniImageNet, TieredImageNet, and other noisy benchmarks, demonstrating the power of low-rank geometry for noise detection and correction in FSL.

General Machine Learning · Unsupervised and Semi-supervised Learning

Zhixin Zhou, Navin Souda, Arash Amini

This paper addresses the problem of linear seriation: recovering the intrinsic order of noisy samples drawn from an unknown one-dimensional manifold embedded in a higher-dimensional space. We propose a multi-stage approach that first robustly estimates local tangent directions using Principal Component Analysis (PCA) on neighborhoods, establishing theoretical consistency for these local estimates. Global orientation consistency of these tangents is then achieved through a spectral relaxation of a pairwise alignment objective. Finally, a globally consistent 1D embedding is computed by solving a carefully formulated linear system (or equivalently, a spectral problem on a derived Laplacian) that aligns the embedding with the oriented local projections. This method effectively leverages local geometric information while ensuring global coherence, producing an ordering robust to noise, curvature, and initial data rotation. We demonstrate its performance on simulated manifold data and discuss the theoretical underpinnings of its core components.

General Machine Learning · Unsupervised and Semi-supervised Learning

Shiyuan Li, Yixin Liu, Yu Zheng, Xiaofeng Cao, Shirui Pan, Heng Tao Shen

Tabular anomaly detection (TAD) aims to identify samples that deviate from the majority in tabular data and is critical in many real-world applications. However, existing methods follow a ``one model for one dataset (OFO)'' paradigm, which relies on dataset-specific training and thus incurs high computational cost and yields limited generalization to unseen domains. To address these limitations, we propose OFA-TAD, a generalist one-for-all (OFA) TAD framework that only requires one-time training on multiple source datasets and can generalize to unseen datasets from diverse domains on-the-fly. To realize one-for-all tabular anomaly detection, OFA-TAD extracts neighbor-distance patterns as transferable cues, and introduces multi-view neighbor-distance representations from multiple transformation-induced metric spaces to mitigate the transformation sensitivity of distance profiles. To adaptively combine multi-view distance evidence, a Mixture-of-Experts (MoE) scoring network is employed for view-specific anomaly scoring and entropy-regularized gated fusion, with a multi-strategy anomaly synthesis mechanism to support training under the one-class constraint. Extensive experiments on 34 datasets from 14 domains demonstrate that OFA-TAD achieves superior anomaly detection performance and strong cross-domain generalizability under the strict OFA setting.

General Machine Learning · Unsupervised and Semi-supervised Learning

Fernando Julio Cendra, Kai Han

Generalized Category Discovery (GCD) aims to identify both known and novel categories within unlabeled data by leveraging a set of labeled examples from known categories. Existing GCD methods primarily depend on semantic labels and global image representations, often overlooking the detailed part-level cues that are crucial for distinguishing closely related categories. In this paper, we introduce PartCo, short for Part-Level Correspondence Prior, a novel framework that enhances category discovery by incorporating part-level visual feature correspondences. By leveraging part-level relationships, PartCo captures finer-grained semantic structures, enabling a more nuanced understanding of category relationships. Importantly, PartCo seamlessly integrates with existing GCD methods without requiring significant modifications. Our extensive experiments on multiple benchmark datasets demonstrate that PartCo significantly improves the performance of current GCD approaches, outperforming most existing methods by bridging the gap between semantic labels and part-level visual compositions, thereby setting new benchmarks for GCD. Code will be made publicly available.

General Machine Learning · Unsupervised and Semi-supervised Learning

Hongyang He, Yan Zhong, Xinyuan Song, Daizong Liu, Xuanyu Liu, Victor Sanchez

Most semi-supervised learning frameworks rely on a single teacher that transfers zero-order supervision through pseudo-labels, constraining the student to imitate categorical outputs without perceiving the loss geometry. This design often leads to unstable optimization and limited generalization under scarce labels. We propose TTN (Two-Teachers Newton-guided Learning), a dual-teacher framework that integrates complementary supervision from MAE and DINOv3 and optimizes the student through a Newton step update. The two teachers provide multi-scale structural and semantic cues whose pseudo-labels and local Hessians are fused by confidence weighting, forming a unified second-order supervision signal. The student updates parameters preconditioned by the fused curvature, enabling stable convergence and geometry-consistent learning. TTN consistently improves over existing single-teacher and consistency-based semi-supervised learning methods on ImageNet, CIFAR-10, SVHN, and STL-10, demonstrating that combining multi-view self-supervised teachers with curvature-guided optimization yields robust and efficient semi-supervised learning.

General Machine Learning · Unsupervised and Semi-supervised Learning

nan cao, Xu Zhao, Teng Zhang

Multi-instance partial-label learning (MIPL) is a recently proposed learning paradigm to address tasks that multi-instance bags are associated with a candidate label set comprising one ground-truth label and several false positive labels. Existing MIPL methods rely on simple instance level information, and can hardly find the key instances under noisy labels. In this paper, we propose a novel algorithm termed AGOPMIPL, i.e., Average Gradient Outer Product based Multi-Instance Partial-Label Learning to address the problem. AGOP derives a data-dependent metric in the embedding space by computing the outer product of classifier gradients, which stretches discriminative feature dimensions and facilitates more accurate key instance identification. Moreover, AGOP aggregates gradient information across all training samples, providing inherent robustness to label noise. Additionally, we introduce a progressive label disambiguation strategy that gradually refine the learning targets. Experimental studies on benchmark and real-world datasets demonstrate the superiority of AGOPMIPL over existing MIPL methods.

General Machine Learning · Unsupervised and Semi-supervised Learning

Long Tang, Keyang Pu, Yingjie Tian

Generalized Zero-Shot Learning (GZSL) suffers from severe seen-class bias, a challenge stemming from the label incompleteness inherent in mixed test distributions. To address this, we propose PUFE, a unified plug-and-play framework that recasts GZSL inference as a Positive-Unlabeled (PU) learning task by treating seen categories as positive and mixed test data as unlabeled. Serving as a seamless post-processing module, PUFE constructs a PU classifier in the semantic space, jointly estimating the seen-class posterior and labeling propensity via Maximum Likelihood Estimation (MLE) within a dual-head network. Furthermore, we introduce an adaptive prototype calibration strategy that leverages high-confidence pseudo-instances—identified by the PU classifier—to explicitly align semantic prototypes with the underlying test distribution. Extensive experiments demonstrate that PUFE consistently mitigates bias and significantly boosts the performance of various embedding-based baselines, yielding gains of up to 11.2 percentage points in the harmonic mean.

General Machine Learning · Unsupervised and Semi-supervised Learning

Kunling Li, Cunqing Hua, Hongyu Zhu, Tianjie Ju, Pengwenlong Gu

Radio Frequency Fingerprint Identification (RFFI) is a foundational pillar of physical-layer security, providing unclonable identity authentication and lightweight defense mechanisms for zero-trust wireless networks. Its practical deployment, however, often occurs in a source-free open-world (SF-OW) setting, characterized by a continuous influx of unregistered devices and privacy constraints that preclude the retention of historical data. In this paper, we formalize SF-OW RFFI task, which manifests a severe stability-plasticity dilemma: intrinsic signal similarity confuses new classes, while source absence precipitates catastrophic forgetting. To address this, we propose Incremental Orthogonal ETF (IO-ETF), a novel neural collapse-inspired framework utilizing output geometry to actively induce parameter separation and isolation. We further devise a Triple-Level Geometric Alignment (TLGA) strategy via semantic optimal transport, manifold progressive anchoring, and reliable subspace retention to stably align unlabeled streams to this geometric skeleton. Experiments on benchmarks demonstrate a superior trade-off between old-class retention and new-class discovery, offering a promising solution for secure access in dynamic networks.

General Machine Learning · Unsupervised and Semi-supervised Learning

Yuan Guo, Wanqi Zhang, Xu Wang

Reconstruction-based multimodal anomaly detection is fundamentally challenged by the one-to-many crossmodal mapping problem, where a single 3D feature may correspond to multiple valid RGB appearances, often leading to collapsed reconstructions and degraded detection performance. We propose *Crossmodal Feature Replacer (CFR)*, a self-supervised ensemble framework that resolves ambiguous crossmodal reconstructions. CFR first learns cyclic mappings between RGB and 3D features while constructing modality-specific and paired memory banks. It then employs an attention-based retrieval network to identify reliable crossmodal candidates. During inference, unreliable reconstructed features are selectively replaced with high-confidence retrieved features, yielding an unambiguous representation for anomaly detection. We evaluate CFR on MVTec 3D-AD and Eyecandies under few-shot settings. Extensive experiments show that CFR consistently outperforms state-of-the-art methods, achieving 92.3 (82.7) 30\% AUPRO and 74.2 (75.9) I-AUROC in the 1-shot setting, demonstrating its effectiveness in addressing crossmodal reconstruction ambiguity.

General Machine Learning · Unsupervised and Semi-supervised Learning

Se Yoon Lee, Jae-kwang Kim

Obtaining high-quality labels is costly, whereas unlabeled covariates are often abundant, motivating semi-supervised inference methods with reliable uncertainty quantification. Prediction-powered inference (PPI) leverages a machine-learning predictor trained on a small labeled sample to improve efficiency, but it can lose efficiency under model misspecification and suffer from coverage distortions due to label reuse. We introduce Machine‑Learning‑Assisted Generalized Entropy Calibration (MEC), a cross‑fitted, calibration‑weighted variant of PPI. MEC improves efficiency by reweighting labeled samples to better align with the target population, using a principled calibration framework based on Bregman projections. This yields robustness to affine transformations of the predictor and relaxes requirements for validity by replacing conditions on raw prediction error with weaker projection‑error conditions. As a result, MEC attains the semiparametric efficiency bound under weaker assumptions than existing PPI variants. Across simulations and a real‑data application, MEC achieves near‑nominal coverage and tighter confidence intervals than CF‑PPI and vanilla PPI.