论文检索

General Machine Learning · Representation Learning

Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis Nicolaou, Yannis Panagakis

Supervised classifier learning has a theoretical optimum — Neural Collapse (NC) — yet standard training does not reach it in practice. We trace this failure to a geometric limitation: cross-entropy is invariant to joint rescaling of features and weights, leaving radial degrees of freedom unconstrained and the loss landscape degenerate. Projecting optimization onto the unit hypersphere eliminates this degeneracy and exposes a hidden equivalence: normalized softmax classification and supervised contrastive learning are conceptually the same, both optimizing angular similarity to class prototypes. We formalize this unification by proving that supervised contrastive learning already produces an optimal classifier during training, the prototype classifier whose weights are given by class-wise feature means, rendering subsequent classifier learning through linear probing redundant. Building on this framework, we identify two computational bottlenecks that slow convergence to NC: the small effective negative set in classifier learning (limited to K class prototypes), and the coupling of competing optimization terms through a shared normalization. We address these with NTCE, which expands the negative set from K classes to M batch instances, and NONL, which normalizes only over negatives to decouple intra-class alignment from inter-class repulsion. Empirically, our methods surpass cross-entropy accuracy on four benchmarks including ImageNet-1K, achieve $\ge$95\% NC across all metrics, and yield consistent gains in transfer learning (+5.5\% mean relative improvement), long-tailed classification (up to +8.7\%), and robustness (lower mCE), while eliminating hours of post-hoc classifier training.

Social Aspects · Accountability, Transparency, and Interpretability

Panagiotis Koromilas, Andreas Demou, James Oldfield, Yannis Panagakis, Mihalis Nicolaou

Sparse autoencoders (SAEs) have emerged as a promising method for interpreting neural network representations by decomposing activations into sparse combinations of dictionary atoms. However, SAEs assume that features combine additively through linear reconstruction, an assumption that cannot capture compositional structure: linear models cannot distinguish whether "Starbucks" arises from the composition of "star" and "coffee" features or merely their co-occurrence. This forces SAEs to allocate monolithic features for compound concepts rather than decomposing them into interpretable constituents. We introduce PolySAE, which extends the SAE decoder with higher-order terms to model feature interactions while preserving the linear encoder essential for interpretability. Through low-rank tensor factorization on a shared projection subspace, PolySAE captures pairwise and triple feature interactions with small parameter overhead (3\% on GPT2). Across four language models and three SAE variants, PolySAE achieves an average improvement of approximately 8\% in probing F1 while maintaining comparable reconstruction error, and produces 2-10$\times$ larger Wasserstein distances between class-conditional feature distributions. Critically, learned interaction weights exhibit negligible correlation with co-occurrence frequency ($r = 0.06$ vs. $r = 0.82$ for SAE feature covariance), suggesting that polynomial terms capture compositional structure, such as morphological binding and phrasal composition, largely independent of surface statistics.

Deep Learning · Foundation Models

Tianbo Wang, Yuqing Ma, Lingyan Meng, Zhange Zhang, Kewei Liao, Jian Yang, Simin Li, Jinyang Guo, Xianglong Liu

Medical Large Vision-Language Models (Med-LVLMs) suffer from severe hallucinations, posing critical safety risks in clinical deployment. Editing LVLM activations has shown promise for mitigating hallucination with minimal cost. However, due to the requirements of medical domain expertise, existing methods struggle to capture imaging manifestations and diagnostic principles that are critical for clinical interpretation, thereby limiting their effectiveness. To address these limitations, we propose the first MEDical-oriented Activation Editing (MEDA) method by integrating Query-decisive Manifestation Steering (QMS) and Principle-driven Diagnosis Induction (PDI) to promote Med-LVLM's expertise elicitation. Specifically, QMS retrieves positive query-decisive imaging manifestations as trusted guidance for activation steering, while PDI constructs positive principle-embedded diagnostic prompts to induce expert-like clinical reasoning. Extensive experiments across six benchmarks and six LVLMs demonstrate that MEDA efficiently improves the response factuality with up to a 10.2\% gain on IU-Xray, while exhibiting strong generalization and few-shot robustness.

Chris Chiu, Simpson Zhang, Mihaela van der Schaar

Emerging agentic marketplaces provide the economic infrastructure for matching and coordinating the large amounts of AI agents used in agentic swarms. Unlike human workers, AI agents can operate on multiple jobs simultaneously, acquire skills rapidly, and labor without wage floors. These differences introduce a new segment of AI labor markets, where AI agents interact with each other at a much higher frequency than human markets. Yet we lack frameworks to understand how such markets behave in light of economic forces that shape labor markets, such as adverse selection and reputation dynamics. To explore this, we introduce AI-Work, a tractable, simulated gig economy where Large Language Model (LLM) agents compete for jobs, develop skills, and adapt their strategies under uncertainty and competitive pressure. Our experiments examine three domains of capabilities that successful agents possess: metacognition (accurate self-assessment of skills), competitive awareness (modeling rivals and market dynamics), and long-horizon strategic planning. Agents with these capabilities consistently achieve higher profits, reputations, and market share than competing agents. Through AI-Work, we hope to provide a foundation to explore the microeconomic properties of AI-only labour markets, and a conceptual framework to study the strategic reasoning capabilities of participating AI agents.

Applications · Everything Else

Jan Melechovsky, Ambuj Mehrish, Abhinaba Roy, Dorien Herremans

Music recordings often suffer from audio quality issues such as excessive reverberation, distortion, clipping, tonal imbalances, and a narrowed stereo image, especially when created in non-professional settings without specialized equipment or expertise. These problems are typically corrected using separate specialized tools and manual adjustments. In this paper, we introduce SonicMaster, the first unified generative model for music restoration and mastering that addresses a broad spectrum of audio artifacts with text-based control. SonicMaster is conditioned on natural language instructions to apply targeted enhancements, or can operate in an automatic mode for general restoration. To train this model, we construct the SonicMaster dataset, a large dataset of paired degraded and high-quality tracks by simulating common degradation types with nineteen degradation functions belonging to five enhancements groups: equalization, dynamics, reverb, amplitude, and stereo. Our approach leverages a flow-matching generative training paradigm to learn an audio transformation that maps degraded inputs to their cleaned, mastered versions guided by text prompts. Objective audio quality metrics demonstrate that SonicMaster significantly improves sound quality across all artifact categories. Furthermore, subjective listening tests confirm that listeners prefer SonicMaster’s enhanced outputs over other baselines. Demo samples are available through https://msonic793.github.io/SonicMaster/

Applications · Computer Vision

Ho Hin Lee, Quan Liu, Shunxing Bao, Yuankai Huo, Bennett Landman

Large kernel convolutions offer a scalable alternative to vision transformers for high-resolution 3D volumetric analysis, yet naïvely increasing kernel size often leads to optimization instability. Motivated by the spatial bias inherent in effective receptive fields (ERFs), we theoretically demonstrate that structurally re-parameterized blocks induce spatially varying learning rates that are crucial for convergence. Leveraging this insight, we introduce Rep3D, a framework that employs a lightweight modulation network to generate receptive-biased scaling masks, adaptively re-weighting kernel updates within a plain encoder architecture. This approach unifies spatial inductive bias with optimization-aware learning, avoiding the complexity of multi-branch designs while ensuring robust local-to-global convergence. Extensive evaluations on five 3D segmentation benchmarks demonstrate that Rep3D consistently outperforms state-of-the-art transformer and fixed-prior baselines.

Deep Learning · Foundation Models

Seunghan Lee

TabPFN has recently gained attention as a foundation model for tabular datasets, achieving strong performance by leveraging in-context learning on synthetic data. However, we find that TabPFN is vulnerable to label shift, often overfitting to the majority class in the training dataset. To address this limitation, we propose DistPFN, the first test-time posterior adjustment method designed for tabular foundation models. DistPFN rescales predicted class probabilities by downweighting the influence of the training prior (i.e., the class distribution of the context) and emphasizing the contribution of the model’s predicted posterior, without architectural modification or additional training. We further introduce DistPFN-T, which incorporates temperature scaling to adaptively control the adjustment strength based on the discrepancy between prior and posterior. We evaluate our methods on over 250 OpenML datasets, demonstrating substantial improvements for various TabPFN-based models in classification tasks under label shift, while maintaining strong performance in standard settings without label shift.

Deep Learning · Foundation Models

Wish Suharitdamrong, Tony Alex, Muhammad Awais, Sara Atito

Foundation models have revolutionized AI, but adapting them efficiently for multimodal tasks, particularly in dual-stream architectures composed of unimodal encoders, such as DINO and BERT, remains a significant challenge. Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) enable lightweight adaptation, yet they operate in isolation within each modality, limiting their ability in capturing cross-modal interactions. In this paper, we take a step in bridging this gap with Cross-Modal Low-Rank Adaptation (CoLA), a novel PEFT framework that extends LoRA by introducing a dedicated inter-modal adaptation pathway alongside the standard intra-modal one. This dual-path design enables CoLA to adapt unimodal foundation models to multimodal tasks effectively, without interference between modality-specific and cross-modal learning. We evaluate CoLA across a range of vision-language (RefCOCO, RefCOCO+, RefCOCOg) and audio-visual (AVE, AVS) benchmarks, where it consistently outperforms LORA, achieving a relative gain of around 3% and 2%, respectively, while maintaining parameter efficiency. Notably, CoLA enables the first multi-task PEFT framework for visual grounding, bridging a key gap in efficient multimodal adaptation.

Deep Learning · Foundation Models

Juntong Chen, Sophie Langer, Johannes Schmidt-Hieber

The recent statistical theory of neural networks focuses on nonparametric denoising problems that treat randomness as additive noise. Variability in image classification datasets does, however, not originate from additive noise but from variation of the shape and other characteristics of the same object across different images. To address this problem, we introduce a tractable model for supervised image classification. While from the function estimation point of view, every pixel in an image is a variable, and large images lead to high-dimensional function recovery tasks suffering from the curse of dimensionality, increasing the number of pixels in the proposed image deformation model enhances the image resolution and makes the object classification problem easier. We introduce and theoretically analyze three approaches. Two methods combine image alignment with a one-nearest neighbor classifier. Under a separation condition, it is shown that perfect classification is possible. The third method fits a convolutional neural network (CNN) to the data. We derive a rate for the misclassification error that depends on the sample size and the complexity of the deformation class. An empirical study corroborates the theoretical findings.

Social Aspects · Safety

Emanuel Tewolde, Xiao Zhang, David Guzman Piedrahita, Vincent Conitzer, Zhijing Jin

It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, according to recent works, the opposite trend appears to be the case: LLMs with stronger reasoning capabilities behave _less_ cooperatively in mixed-motive games such as the prisoner's dilemma and in public goods settings. Indeed, our experiments show that recent models---with or without reasoning enabled---consistently defect on the other players in single-shot social dilemmas. To tackle this safety concern, we study game-theoretic mechanisms that are designed to enable cooperative outcomes between rational agents _in equilibrium_. Across four social dilemmas testing distinct components of robust cooperation, we evaluate under the following mechanisms: (1) repeating the game for many rounds, (2) reputation systems, (3) third-party mediators to delegate decision making to, and (4) contract agreements for outcome-conditional payments between players. Among our findings, we establish that contracting and mediation are most effective in achieving cooperative outcomes between capable LLM models, and that repetition-induced cooperation deteriorates drastically when co-players vary. Moreover, we demonstrate that these cooperation mechanisms become _more effective_ with higher pressures to optimize for one own's utility.

General Machine Learning · Online Learning, Active Learning and Bandits

Ganghua Wang, Yuwei Cheng, Haifeng Xu

The probabilistic bisection algorithm (PBA) extends the classical binary search to settings with noisy responses, and is a foundational algorithm commonly used in basic problems such as root-finding. Despite its strong empirical success, its theoretical property, particularly the convergence rate, remains unclear. This paper establishes that PBA converges at a geometric rate, providing a rigorous justification for its empirical efficiency. Notably, this rate is optimal in the sense that it matches the performance of classical binary search under noiseless responses. The core of our analysis lies in directly characterizing the dynamics of PBA queries, which had not been examined in the prior literature. We show that the queries oscillate around the truth but steadily draw closer, thus leading to an estimator that rapidly concentrates on the truth. Beyond resolving the long-standing question of PBA’s convergence, our developed techniques offer new tools for analyzing PBA's dynamics, which may be of independent interest.

Zichao Yue, Zhiru Zhang

Pre-propagation graph neural networks (PP-GNNs) decouple node feature propagation from transformation: graph diffusion is performed once as preprocessing, and training reduces to dense per-node transformations. This design enables mini-batch training without inter-node dependencies, avoids repeated sparse matrix--matrix multiplications, and better matches modern accelerators optimized for dense compute. However, their expressivity remains unclear, and empirical results show a gap between PP-GNNs and their message-passing counterparts on commonly used graph benchmarks, especially heterophilic ones. In this paper, we propose a suite of robust graph diffusion operators for preprocessing and a few-shot hidden-state re-propagation scheme during training. Our methods improve the validation and test accuracy of PP-GNNs, enabling them to match the accuracy of message-passing GNNs while maintaining training efficiency.

Applications · Robotics

Kaiwen Xue, Tao Wei, Zhonghong Ou, Guoxin Zhang, Kaoyan Lu, Yu Feng, Yifan Zhu, Haoran Luo

Multimodal large language models (MLLMs) have shown strong potential for building embodied agents, yet embodied geo-localization remains underexplored due to the lack of fine-grained evaluation. We introduce ERGeoBench, a large-scale benchmark for vision-driven embodied geo-localization. ERGeoBench evaluates models under three progressively interactive settings—single-view, multi-view, and embodied-view—where agents actively acquire observations through sequential viewpoint changes. The benchmark comprises 2,207 globally distributed street-view panoramas and assesses four core capability dimensions: foundational perception, spatial awareness, commonsense reasoning, and geo-localization. Extensive evaluations of leading proprietary and open-source MLLMs reveal that while current models perform well in high-level semantic geo-localization, they struggle with low-level perceptual operations and maintaining spatial consistency across views. Notably, geo-localization performance exhibits a strong positive correlation with the other three capability dimensions, indicating that accurate localization emerges from robust perception, coherent spatial reasoning, and sound commonsense understanding. Overall, ERGeoBench provides a unified and diagnostic framework for advancing human-like embodied geo-localization.

Applications · Language, Speech and Dialog

Minchan Kwon, Sung Hyun Baek, Minseo Kim, Jaemyung Yu, Dongyoon Han, Junmo Kim

Large Language Model Red-Teaming, which proactively identifies vulnerabilities of large language models, is an essential process for ensuring safety. Finding effective and diverse attacks in red team activities is important, but achieving both is challenging. Generative Flow Networks (GFN) that perform distribution matching are a promising method, but they are notorious for training instability and mode collapse. In particular, unstable reward functions in red team activities accelerate mode collapse. We propose Stable-GFN (S-GFN), which eliminates Z estimation in GFN and reduces training instability. S-GFN avoids Z-estimation through pairwise comparisons and employs a robust masking methodology against noisy rewards. Additionally, we propose a fluency stabilizer to prevent the model from getting stuck in local optima that produce gibberish. S-GFN provides more stable training while maintaining the optimal policy of GFN. We demonstrate the overwhelming attack performance and diversity of S-GFN across various settings.

Social Aspects · Everything Else

Jiangwei Chen, Xinyuan Niu, Rachael Hwee Ling Sim, Zhengyuan Liu, Nancy Chen, Bryan Kian Hsiang Low

Machine unlearning aims to remove the influence of specific training samples due to privacy, copyright or bias concerns. Multi-objective unlearning seeks to ensure the effective forgetting of such samples while preserving the utility of the unlearned model. Existing multi-objective unlearning methods typically optimize a weighted combination of the objectives. They provide no guarantee that any of the objectives can achieve a required performance and do not consider the similarity between the forget data and the remaining retain data. In this work, we quantify how hard it is to reconcile the conflicting objectives arising from overlapping data and provide conditions under which collateral forgetting is unavoidable, that is, when improving forget quality forces retain utility degradation. Utilizing this hardness measure, we propose a hardness-aware multi-objective unlearning algorithm (HAMU) that adapts the unlearning updates based on the per-iteration hardness. Our algorithm is applicable to non-convex models and is easily parallelizable, making it readily deployable in real-world scenarios. We empirically demonstrate HAMU's superior performance over baselines on both image and text datasets using large-scale models.

Optimization · Everything Else

Shenyang Deng, Zhuoli Ouyang, Ruochen Jin, Tianyu Pang, Zihang Liu, Shuhua Yu, Yaoqing Yang

Preconditioned adaptive methods have gained significant attention for training deep neural networks, as they capture rich curvature information. The central challenge in this field lies in balancing preconditioning effectiveness with computational efficiency of implementing the preconditioner. Among recent advances, \textsc{MUON} stands out by using Newton-Schulz iteration to obtain preconditioned updates without explicitly constructing the preconditioning matrix. In this paper, we introduce \textsc{RMNP} (Row Momentum Normalized Preconditioning), an optimizer that replaces Newton-Schulz iteration with a simple row-wise $\ell_2$ normalization operation, motivated by the empirically observed diagonal block structure of the Transformer layerwise Hessian. This substitution reduces the per-iteration complexity from $\mathcal{O}(mn\cdot\min(m,n))$ to $\mathcal{O}(mn)$ for an $m\times n$ weight matrix while maintaining comparable optimization performance. Theoretically, we establish convergence guarantees for \textsc{RMNP} in the non-convex setting that match recent results for \textsc{Muon} optimizers, achieving the information-theoretic minimax optimal complexity. Extensive experiments on large language model pretraining show that \textsc{RMNP} delivers competitive optimization performance compared with \textsc{Muon} while substantially reducing preconditioning process wall-clock time. Our code is available at \href{https://anonymous.4open.science/r/RMNP-317C/}{link}.

Deep Learning · Attention Mechanisms

Elon Litman, Gabe Guo

We generalize the attention mechanism by viewing it through the lens of Entropic Optimal Transport, revealing that standard attention corresponds to a transport problem regularized by an implicit uniform prior. We introduce Generalized Optimal transport Attention with Trainable priors (GOAT), a new attention mechanism that replaces this naive assumption with a learnable, continuous prior. This prior maintains full compatibility with optimized kernels such as FlashAttention. GOAT also provides an EOT-based explanation of attention sinks and materializes a solution for them, avoiding the representational trade-offs of standard attention. Finally, by absorbing spatial information into the core attention computation, GOAT learns an extrapolatable prior that combines the flexibility of learned positional embeddings with the length generalization of fixed encodings.

Applications · Health / Medicine

Yipan Wei, Wenke Huang, Yapeng Li, He Li, Qixin Zhang, Mang Ye, Bo Du

Federated Prompt Learning (FPL) adapts Vision-Language Models to privacy-sensitive medical imaging, typically via a textual tuning paradigm that assumes the frozen visual encoder provides a discriminative feature geometry. We argue this assumption breaks down in medical settings, leading to two geometric pathologies: (1) Intra-client: Medical Manifold Collapse, where high morphological similarity reduces the effective rank of visual features; and (2) Inter-client: Medical Topological Misalignment, where heterogeneous acquisition protocols induce inconsistent geometry across clients. To address these, we propose FedMAP, which shifts the paradigm to Visual Manifold Anchoring. FedMAP utilizes an LLM-derived codebook as a client-invariant synchronization signal to restructure the visual space, via Manifold Semantic Anchoring (MSA) and Topology Structural Alignment (TSA) to enforce consistent inter-class relations. Experiments on FedISIC, FedCamelyon17, and a private ultrasound dataset show that FedMAP consistently outperforms state-of-the-art methods, especially in high-noise regimes where manifold collapse is most severe.

Yidong Wang, Xin Wang, Cunxiang Wang, Junfeng Fang, Qiufeng Wang, Jianing Chu, Xuran Meng, Shu-Xun Yang, Andrew Feng, Libo Qin 等

Self-Rewarding Language Models propose an architecture in which the Large Language Models(LLMs) both generates responses and evaluates its own outputs via LLM-as-a-Judge prompting, dynamically improving its generative capabilities through iterative Direct Preference Optimization (DPO). However, our analysis reveals a critical limitation in existing self-rewarding paradigms: the synchronized improvement of chosen and rejected responses progressively narrows the representational difference between contrasting samples, undermining effective preference learning. We propose Temporal Self-Rewarding Language Models that strategically coordinate past, present, and future model generations to sustain learning signals. Our dual-phase framework introduces: (1) Anchored Rejection - fixing rejected responses using the past initial model's outputs and (2) Future-Guided Chosen - dynamically curating chosen samples using next-generation model predictions. Extensive experiments across three model families (Llama, Qwen, Mistral) and different model sizes (Llama3B/8B/70B) demonstrate significant improvements when trained with our method compared to self-rewarding using same computation resources. For example, Llama3.1-8B reaches a 29.44 win rate on AlpacaEval 2.0 with our method, outperforming the self-rewarding baseline (19.69) by 9.75. Notably, our method also demonstrates superior out-of-distribution generalization across mathematical reasoning (GSM8K), knowledge-based QA (ARC, TruthfulQA), and code generation (HumanEval) tasks, even though we do not specifically collect such training data. The generality of our temporal strategy is further validated by its benefits when extended to online reinforcement learning settings on mathematical reasoning tasks.

Deep Learning · Attention Mechanisms

Junchen Liu, Sven Elflein, Or Litany, Zan Gojcic, Ruilong Li

Test-time training (TTT) in transformers is commonly interpreted as a form of online meta-learning that memorizes a key–value mapping at test time. However, our analysis reveals multiple phenomena that contradict this memorization-based interpretation. Motivated by these findings, we revisit the formulation of TTT and show that a broad class of TTT architectures can be expressed as a form of learned linear attention operator. Beyond explaining previously puzzling model behaviors, this perspective yields multiple practical benefits: it enables principled architectural simplifications, admits fully parallel formulations that preserve performance while improving efficiency, and provides a systematic reduction of diverse TTT variants to a standard linear attention form. Overall, our results reframe TTT not as test-time memorization, but as learned linear attention with enhanced representational capacity.