论文检索

Applications · Health / Medicine

Bonjae Ku, Seeun Kim, Yubeen Kim, Hahnbeom Park, Chaok Seok

Proteins are inherently dynamic, with biological functions often emerging from transitions between multiple conformational states. While recent breakthroughs have largely addressed the static structure prediction problem, a systematic benchmark is absent to demonstrate how well current models capture functionally relevant dynamics. We introduce ProMiSE, the first benchmark that provides both a dataset and evaluation scheme, based on native biological assemblies and integrating major conformational change mechanisms—intrinsic, ligand-induced, and protein-induced—within a single curated dataset. We conducted a comprehensive evaluation of state-of-the-art structure prediction models, including AlphaFold3 and recent generative approaches. Our findings reveal that current models exhibit a limited ability to sample intrinsic dynamics and are often insensitive to biological context in induced scenarios. We further investigate whether these multi-state prediction biases are associated with multiple sequence alignment (MSA) signals or training data distributions, while analyzing internal model representations throughout the model to identify where these biases arise. Ultimately, ProMiSE benchmarks limitations in conformational diversity and biological relevance, enabling improved multi-state and dynamics-aware modeling.

Deep Learning · Generative Models and Autoencoders

Seunghyeok Shin, Minwoo Kim, Dabin Kim, Hongki Lim

Diffusion posterior sampling conditions diffusion priors on measurements, but data-consistency updates are typically scaled by hand-tuned guidance weights and can destabilize sampling under stiff, operator-dependent curvature. We replace scalar guidance with a per-noise-level damped Gauss--Newton correction computed in diffusion-state coordinates. The correction pulls likelihood gradients back through the denoiser, uses a one-sided curvature model that avoids forward denoiser Jacobians, and applies diffusion-calibrated rank-one damping aligned with the denoiser residual. Each correction is solved with matrix-free GMRES using automatic differentiation, and sampling proceeds with a variance-preserving Langevin transition with a closed-form drift/noise split. Aside from compute-budget choices ($T$ diffusion steps and $K$ Krylov iterations), the method has a single damping hyperparameter ($\lambda_{\mathrm{id}}$), kept nearly unchanged across tasks. On FFHQ and ImageNet across inverse problems, it achieves competitive PSNR/SSIM/LPIPS while running markedly faster than most of the compared baselines; on accelerated MRI reconstruction, it achieves the best PSNR/SSIM among the compared baselines.

Probabilistic Methods · Gaussian Processes

Marcus Noack, Mark Risser, HENGRUI LUO, Vardaan Tekriwal, Ronald Pandolfi

Despite a large corpus of recent work on scaling up Gaussian processes, a stubborn trade-off between computational speed, prediction and uncertainty quantification accuracy, and customizability persists. This is because the vast majority of existing methodologies exploit various levels of approximations that lower accuracy and limit the flexibility of kernel and noise-model designs --- an unacceptable drawback at a time when expressive non-stationary kernels are on the rise in many fields. Here, we propose a methodology we term \emph{gp2Scale} that scales exact Gaussian processes to more than 10 million data points without relying on inducing points, kernel interpolation, or neighborhood-based approximations, and instead leveraging the existing capabilities of a GP: its kernel design. Highly flexible, compactly supported, and non-stationary kernels lead to the identification of naturally occurring sparse structure in the covariance matrix, which is then exploited for the calculations of the linear system solution and the log-determinant for training. We demonstrate our method's functionality on several real-world datasets and compare it with state-of-the-art approximation algorithms. Although we show superior approximation performance in many cases, the method's real power lies in its agnosticism toward arbitrary GP customizations --- core kernel design, noise, and mean functions --- and the type of input space, making it optimally suited for modern Gaussian process applications.

Optimization · Convex

Xiyuan Wei, Linli Zhou, Bokun Wang, Chih-Jen Lin, Tianbao Yang

This paper studies optimization for a family of problems termed **compositional entropic risk minimization**, in which each data's loss is formulated as a Log-Expectation-Exponential (Log-E-Exp) function. The Log-E-Exp formulation serves as an abstraction of the Log-Sum-Exponential (LogSumExp) function when the explicit summation inside the logarithm is taken over a gigantic number of items and is therefore expensive to evaluate. While entropic risk objectives of this form arise in many machine learning problems, existing optimization algorithms suffer from several fundamental limitations including non-convergence, numerical instability, and slow convergence rates. To address these limitations, we propose a geometry-aware stochastic algorithm, termed **SCENT**, for the dual formulation of entropic risk minimization cast as a min--min optimization problem. The key to our design is a **stochastic proximal mirror descent (SPMD)** update for the dual variable, equipped with a Bregman divergence induced by a negative exponential function that faithfully captures the geometry of the objective. Our main contributions are threefold: (i) we establish an $O(1/\sqrt{T})$ convergence rate of the proposed SCENT algorithm for convex problems; (ii) we theoretically characterize the advantages of SPMD over standard SGD update for optimizing the dual variable; and (iii) we demonstrate the empirical effectiveness of SCENT on extreme classification, partial AUC maximization, contrastive learning and distributionally robust optimization, where it consistently outperforms existing baselines.

Theory · Learning Theory

Yuanfan Li, Xiyuan Wei, Tianbao Yang, Yiming Ying

Contrastive representation learning (CRL) underpins many modern foundation models. Despite recent theoretical progress, existing analyses suffer from several key limitations: (i) the statistical consistency of CRL remains poorly understood; (ii) available generalization bounds deteriorate as the number of negative samples increases, contradicting the empirical benefits of large negative sets; and (iii) the retrieval performance of CRL has received limited theoretical attention. In this paper, we develop a unified statistical learning theory for CRL. For downstream tasks, we evaluate retrieval quality using an AUC-type population criterion and show that the contrastive loss is *statistically consistent* with optimal ranking. We further establish a *calibration-style inequality* that quantitatively relates excess contrastive risk to excess retrieval suboptimality. For upstream training, we study both supervised and self-supervised contrastive objectives and derive generalization bounds of order $O(1/m + 1/\sqrt{n})$ and $O(1/\sqrt{m} + 1/\sqrt{n})$, respectively, where $m$ denotes the number of negative samples and $n$ the number of anchor points. These bounds not only explain the empirical advantages of large negative sets but also reveal an explicit trade-off between $m$ and $n$. Extensive experiments on large-scale vision--language models corroborate our theoretical predictions.

Applications · Time Series

Linxiao Yang, Xue Jiang, Gezheng Xu, Tian Zhou, Min Yang, Zhaoyang Zhu, Linyuan Geng, Zhipeng Zeng, Qiming Chen, Xinyue Gu 等

Transformers enable in-context learning (ICL) for rapid, gradient-free adaptation in time series forecasting, yet most ICL-style approaches rely on tabularized, hand-crafted features, while end-to-end sequence models lack inference-time adaptation. We bridge this gap with a unified framework, Baguan-TS, which integrates the raw-sequence representation learning with ICL, instantiated by a 3D Transformer that attends jointly over temporal, variable, and context axes. To make this high-capacity model practical, we tackle two key hurdles: (i) calibration and training stability, improved with a feature-agnostic, target-space retrieval-based local calibration; and (ii) output oversmoothing, mitigated via context-overfitting strategy. On public benchmark with covariates, Baguan-TS consistently outperforms established baselines, achieving the highest win rate and significant reductions in both point and probabilistic forecasting metrics. Further evaluations across diverse real-world energy datasets demonstrate its robustness, yielding substantial improvements.

Theory · Online Learning and Bandits

Tomer Gafni, Garud Iyengar, Assaf Zeevi

We consider an online learning problem in environments with multiple change points. In contrast to the single change point problem that is widely studied using classical "high confidence" detection schemes, the multiple change point environment presents new learning-theoretic and algorithmic challenges. Specifically, we show that classical methods may exhibit catastrophic failure (high regret) due to a phenomenon we refer to as endogenous confounding. To overcome this, we propose a new class of learning algorithms dubbed Anytime Tracking CUSUM (ATC). These are horizon-free online algorithms that implement a selective detection principle, balancing the need to ignore "small" (hard-to-detect) shifts, while reacting "quickly" to significant ones. We prove that the performance of a properly tuned ATC algorithm is nearly minimax-optimal; its regret is guaranteed to closely match a novel information-theoretic lower bound on the achievable performance of any learning algorithm in the multiple change point problem. Experiments on synthetic as well as real-world data validate the aforementioned theoretical findings.

Social Aspects · Privacy

Zoë R Bell, Anvith Thudi, Olive Franzese-McLaughlin, Nicolas Papernot, Shafi Goldwasser

Training with differential privacy (DP) guarantees dataset members that they cannot be identified by users of the released model. However, those data providers, and, in general, the public, lack methods to efficiently verify that models trained on their data satisfy DP guarantees. The amount of compute needed to verify DP guarantees for current algorithms scales with the amount of computation required to train the model. In this paper we design the first DP algorithm with near optimal privacy-utility trade-offs but whose DP guarantees can be verified cheaper than training. We focus on DP stochastic convex optimization (DP-SCO), where optimal privacy-utility trade-offs are known. Here we show we can obtain tight privacy-utility trade-offs by privately minimizing a series of regularized objectives and only using the standard DP composition bound. Crucially, this method can be verified with much less compute than training. This leads to the first known DP-SCO algorithm with near optimal privacy-utility whose DP verification scales better than training cost, significantly reducing verification costs on large datasets.

Applications · Computer Vision

Lin Fu, Zheyuan Yang, Yang Wang, Tingyu Song, Arman Cohan, Yilun Zhao

We introduce VideoKR, the first large-scale training corpus specifically designed to strengthen knowledge- and reasoning-intensive video understanding. It comprises 430K video reasoning examples over 126K newly collected, CC-licensed, expert-domain videos. We develop an expert-in-the-loop, skill-oriented example generation pipeline that targets progressively deeper video reasoning capabilities while ensuring the difficulty, diversity, and reliability of both the examples and their CoT rationales. We also curate VideoKR-Eval, a new expert-annotated benchmark where questions require genuine video understanding and knowledge-intensive reasoning. Our experiments show that, under a standard SFT→GRPO pipeline, models post-trained on VideoKR already outperform prior post-training approaches on both general and knowledge-intensive video reasoning benchmarks, highlighting data design as a key driver of progress in video reasoning. We further conduct comprehensive ablations to isolate the contributions of VideoKR, providing actionable insights for future work.

Applications · Computer Vision

Yuying Chen, Liu, Linyan Jiang, Qifan Gao, Xianguo Zhang, Jianhou Gan, Wenqi Ren

Recent advances in video diffusion models have demonstrated remarkable generative capability, yet adapting these large pretrained text-to-video (T2V) models to video super‑resolution (VSR) typically encounters challenges, such as artifacts introduced by complex degradations in real-world scenarios and compromised fidelity due to the strong generative capacity of the powerful T2V models. We present WEVSR, a novel approach that adapts a pretrained flow-matching video diffusion transformer to RealVSR. First, we design a task-oriented adaptation strategy that leverages timestep sampling and noise augmentation to enhance detail restoration while preserving structural stability. Second, we propose a lightweight multi-level discrete wavelet transform (DWT) front-end for the VAE encoder, injecting explicit frequency priors into the latent space without modifying the pretrained decoder. Extensive experiments across multiple RealVSR benchmarks demonstrate that WEVSR achieves state-of-the-art performance against existing approaches.

Applications · Chemistry, Physics, and Earth Sciences

Matteo Rossi, Ryan Pederson, Miles Wang-Henderson, Benjamin Kaufman, Edward Williams, Carl Underkoffler, Owen Howell, Adrian Layer, Stephan Thaler, Narbe Mardirossian 等

We present CoarseBind, a foundation model for protein-ligand structure and binding affinity prediction that achieves 26$\times$ faster inference than state-of-the-art methods while improving affinity prediction accuracy by up to 20\%. Current deep learning approaches to structure-based drug design rely on expensive all-atom diffusion to generate 3D coordinates, creating inference bottlenecks that render large-scale compound screening computationally intractable. We challenge this paradigm with the hypothesis: full all-atom resolution is unnecessary for accurate small molecule pose and binding affinity prediction. CoarseBind tests this hypothesis through a coarse pocket-level representation (protein C$_\beta$ atoms and ligand heavy atoms only) within a multimodal architecture combining pretrained molecular encoders and ESM-2 protein embeddings that learns rich structural representations, which are used in a diffusion-free optimization module for pose generation and a binding affinity likelihood prediction module. On structure prediction benchmarks, CoarseBind matches diffusion-based baselines in ligand pose accuracy. For binding affinity, CoarseBind outperforms Boltz-2 by 16-20\% in Pearson correlation on both a public benchmark (CASP16) and a diverse private dataset (18 assays). The affinity module also provides well-calibrated uncertainty estimates, addressing a critical gap in compound prioritization for drug discovery. Furthermore, this module enables a continual learning framework and a hedged batch selection strategy that, in simulated drug discovery cycles, achieves 6$\times$ greater affinity improvement over greedy approaches.

Deep Learning · Graph Neural Networks

Maysam Behmanesh, Erkan Turan, Maks Ovsjanikov

Graph alignment, the problem of identifying corresponding nodes across multiple graphs, is fundamental to numerous applications. Most existing unsupervised methods embed node features into latent representations to enable cross-graph comparison without ground-truth correspondences. However, these methods suffer from two critical limitations: the degradation of node distinctiveness due to oversmoothing in GNN-based embeddings, and the misalignment of latent spaces across graphs caused by structural noise, feature heterogeneity, and training instability, ultimately leading to unreliable node correspondences. Our key insight is that rather than balancing these objectives purely in the feature space, a unified framework can simultaneously filter in both feature and map spaces. We propose a novel framework that employs a dual-pass encoder to inject high-frequency discriminability into node features, paired with a geometry-aware functional map module that operates on the correspondence itself. This functional map module learns bijective and isometric transformations that align latent spaces while acting as a low-pass filter on correspondences, enforcing smoothness and robustness as a structural prior in the map space. Extensive experiments on graph benchmarks demonstrate that our method consistently outperforms existing unsupervised alignment baselines, exhibiting superior robustness to structural inconsistencies and challenging alignment scenarios.

Applications · Computer Vision

Qunzhong Wang, Jie Liu, Jiajun Liang, Yuanxing Zhang, Yilei Jiang, Yaozhi ZHENG, Xintao Wang, Pengfei Wan, Xiangyu Yue, Jiaheng Liu

Recent advancements in multimodal reward models (RMs) have substantially improved post-training for visual generative models. However, current RMs face inherent limitations: **(1)** visual inputs consume large context budgets, forcing fewer frames and causing a loss of details; and **(2)** all visual information is packed into the initial prompt, exacerbating forgetting during chain-of-thought reasoning. To overcome these issues, we introduce **VR-Thinker**, a thinking-with-image RM equipped with visual reasoning operation and a configurable visual memory window. This allows the RM to actively acquire visual evidence, improving reasoning fidelity and reliability. We activate visual reasoning via a reinforcement fine-tuning pipeline: **(i)** **Cold Start** with curated visual chain-of-thought data to distill basic operation formatting; **(ii)** select samples with correct judgments, then conduct **Rejection sampling Fine-Tuning** on these high-quality traces to further enhance reasoning; and **(iii)** apply **Group Relative Policy Optimization** (GRPO) to strengthen reasoning. Our approach delivers state-of-the-art accuracy among open-source models on video preference benchmarks: a 7B **VR-Thinker** achieves 80.5\% on VideoGen Reward, 82.3\% on GenAI-Bench, and 75.6\% on MJ-Bench-Video.

Applications · Computer Vision

Ghadi Nehme, Yanxia Zhang, Dule Shu, Matthew Klenk, Faez Ahmed

Generating high-fidelity 3D geometries that satisfy specific parameter constraints has broad applications in design and engineering. However, current methods typically rely on large training datasets and struggle with controllability and generalization beyond the training distributions. To overcome these limitations, we introduce LAMP (Linear Affine Mixing of Parametric shapes), a data-efficient framework for controllable and interpretable 3D generation. LAMP first aligns signed distance function (SDF) decoders by overfitting each exemplar from a shared initialization, then synthesizes new geometries by solving a parameter-constrained mixing problem in the aligned weight space. To ensure robustness, we further propose a safety metric that detects geometry validity via linearity mismatch. We evaluate LAMP on two 3D parametric benchmarks: DrivAerNet++ and BlendedNet. We found that LAMP enables (i) controlled interpolation within bounds with as few as 100 samples, (ii) safe extrapolation by up to 100\% parameter difference beyond training ranges, (iii) physics performance-guided optimization under fixed parameters. LAMP significantly outperforms conditional autoencoder and Deep Network Interpolation (DNI) baselines in both extrapolation and data efficiency. Our results demonstrate that LAMP advances controllable, data-efficient, and safe 3D generation for design exploration, dataset generation, and performance-driven optimization.

Deep Learning · Self-Supervised Learning

Faris Chaudhry

We develop a geometric theory of projection heads in self-supervised learning by interpreting the head as a trainable metric on the backbone representation manifold. Our analysis reveals that head curvature and architectural asymmetry induce negative eigenvalues of the Hessian at collapsed equilibria in networks with smooth activation functions, yielding a destabilization mechanism which explains collapse avoidance in non-contrastive methods. We further show that linear heads perform implicit subspace whitening under induced metric geometry, while nonlinear heads adapt local metrics to satisfy the specific topological constraints of the loss. Finally, we characterize how metric degeneracy governs the information-invariance trade-off in learned representations. Our results apply to both contrastive and non-contrastive objectives including InfoNCE, BYOL, SimSiam, and decorrelation-based methods, demonstrating that the projection head acts as a universal geometric buffer that decouples the semantic backbone from the rigid constraints of the training objective.

Optimization · Discrete and Combinatorial Optimization

Shengyu Feng, Tarun Suresh, Yiming Yang

Neural solvers have recently emerged as powerful tools for combinatorial optimization (CO). Among them, diffusion models have shown strong promise due to their ability to capture highly multimodal solution distributions in CO through iterative generative processes. However, training diffusion models typically requires large collections of near-optimal solutions, which limits their scalability and generalization. We address this fundamental challenge by extending adjoint matching, a powerful unsupervised diffusion training framework based on chain-rule–style gradient propagation in continuous spaces, to discrete combinatorial domains. Our approach resolves the broken-gradient issue inherent to discrete data and unifies local and global training objectives within a single principled framework. Empirically, our method consistently outperforms existing unsupervised baselines and achieves performance comparable to supervised diffusion models.

Deep Learning · Generative Models and Autoencoders

Jianyuan Zhong, Wang Kaibo, Ding Ding, Zijin Feng, Haoli Bai, Yang Xiang, Jiacheng Sun, Qiang Xu

Diffusion Large Language Models (dLLMs) often exhibit severe instability during Group Relative Policy Optimization (GRPO) training, limiting the effectiveness of reinforcement learning for improving reasoning capabilities. In dLLMs, the importance ratios used by GRPO are derived from finite-sample estimates rather than exact likelihoods, making them inherently noisy. In this paper, we show that GRPO is highly sensitive to this noise, which drives training instability. Through theoretical analysis and empirical evidence, we identify a self-reinforcing instability loop in which noisy importance ratios induce gradient spikes and policy drift, further amplifying future importance ratio estimation variance. To address this issue, we propose StableDRL, a novel reinforcement learning framework for dLLMs. StableDRL stabilizes training via (i) unconditional clipping to suppress outlier-induced gradient spikes, and (ii) self-normalization to constrain gradients within the convex hull of per-sample updates. We further extend StableDRL to block-wise diffusion models via a staircase attention mechanism. StableDRL is the first method that enables stable, full-parameter reinforcement learning for dLLMs. It achieves the state-of-the-art performance, outperforming prior best full-attention baselines by 6% on MATH500 and block-diffusion baselines by 25.6% on AIME.

Applications · Computer Vision

Bin Lei, Nuo Xu, Ali Payani, Mingyi Hong, Chunhua Liao, Yu Cao, Caiwen Ding

Multimodal large language models (MLLMs) have markedly expanded the competence of graphical user-interface (GUI) systems, propelling them beyond controlled simulations into complex, real-world environments across diverse platforms. However, practical usefulness is still bounded by the reliability of visual grounding, i.e., mapping textual references to exact on-screen elements. This limitation prevents the system from accurately performing pointer-level actions such as clicking or dragging. To address it, we introduce GUI-Spotlight -- a model trained for image-grounded reasoning that dynamically invokes multiple specialized tools to iteratively narrow its focus to the relevant region of the screen, thereby substantially improving visual grounding accuracy. On the ScreenSpot-Pro benchmark, GUI-Spotlight trained with only 18.5K training samples achieves 52.8\% accuracy, surpassing V2P-7B (50.6\% with 9.6M training samples) and GTA-1-7B (50.1\% with 1.56M training samples).

Deep Learning · Everything Else

Songsong Ouyang, Le Wu, Yingying Zhu

Cross-view geo-localization (CVGL) aims to retrieve the corresponding satellite image given a street query and is critical for autonomous navigation. Although recent methods perform well on benchmarks, they often fail to generalize to unseen environments. A key limitation is the use of contrastive learning, which assigns equal labels to all negative samples and induces similarity-amplified repulsion. But should all negatives be treated equally? In CVGL, semi-positive samples that are geographically proximate to the positive often share important semantic cues. Treating them as ordinary negatives forces the model to overfit noise, leading to a collapse in generalization. To address this issue, we propose an uncertainty-aware framework grounded in Deep Evidential Regression (DER), modeling the Normal-Inverse-Gamma (NIG) distribution as a conjugate prior to quantify environmental complexity $u$ in a single forward pass. The estimated $u$ adaptively softens labels for hard negatives in Soft InfoNCE, mitigating excessive repulsion on semi-positive samples. An Uncertainty Head with cls-to-spatial cross-attention and attention statistics is designed to accurately fit the NIG distribution. Extensive experiments demonstrate state-of-the-art performance, including an average 18\% R@1 improvement in zero-shot cross-dataset transfer, filling the critical gap between laboratory benchmarks and robust real-world deployment.

Reinforcement Learning · Everything Else

Xiyue Peng, Lingkai Zu, Ziyu Shao, Xin Liu

We study episodic constrained Markov decision processes (CMDPs) with linear function approximation, where the goal is to achieve strong regret and constraint violation guarantees without allowing error cancellations. Unlike the existing work, which focuses on either tabular CMDP or model-based reinforcement learning methods. We propose a model-free policy APMPO that achieves near-optimal $\widetilde{O}(\sqrt{K})$ strong regret and strong constraint violation with Slater's condition (or strict feasibility assumption), where $K$ is the total number of episodes. It matches the best-known rates without requiring any prior knowledge of the feasibility gap reported in prior model-based work for tabular CMDPs. Besides, APMPO achieves $\widetilde{O}(K^{\frac{3}{4}})$ strong regret and $\widetilde{O}(K^\frac{3}{4})$ strong constraint violation without Slater's condition. To the best of our knowledge, this is the first sublinear result of CMDP w.r.t. the strong metrics without Slater's condition. APMPO achieves these results by a novel and adaptive design of a violation-aware penalty and learning rates to balance the strong regret and constraint violation, which is quite different from the (regularized) primal-dual methods imposing constraints via dual penalty in the literature. The experiments show APMPO significantly outperforms the strong baselines, which justify our design and theoretical performance.