Denoising Diffusion Codebook Models (DDCM) have emerged as a promising framework for zero-shot image compression by replacing stochastic sampling with discrete selection from a reproducible Gaussian codebook. By greedily picking noise vectors that best match the target image, DDCM encodes the generative trajectory into a compact sequence of indices for image compression. However, the target-guided noise selection provides only coarse guidance as the discrepancy between the denoiser's output and the target image is substantial. This discrepancy inevitably results in trajectory drift, necessitating massive codebooks to span the enlarged search space and extensive inference steps to gradually compensate for the deviation. In this paper, we propose the Trajectory-Consistent Diffusion Codebook Model (TC-DDCM) to address these inefficiencies. Rather than approximating the distant target, TC-DDCM navigates the generative process by strictly tracking a pre-computed inversion trajectory. This paradigm shift yields two critical advantages: 1) By minimizing the deviation to the proximal reference state along the trajectory, the search space is drastically shrunk, resulting in significantly smaller codebooks. 2) By aligning the generative steps with inversion methods, TC-DDCM achieves optimal reconstruction in a few inference steps. Extensive experiments demonstrate that TC-DDCM significantly outperforms state-of-the-art zero-shot methods in rate-distortion-perception performance at ultra-low bitrates, making zero-shot diffusion-based compression practical.
论文检索
输入标题、作者或关键词，从 2,132 篇学术成果中精准定位
Many scientific and engineering applications require estimating unknown parameters from experimentally observable data -- an inverse problem that is inherently challenging due to nonlinearity, noise, and ill-posedness. In this paper, we propose an active diffusion-based inverse problem solver. A diffusion model is trained to learn the mapping between the parameter space and the observable space. By iteratively detecting and correcting model misspecification through posterior uncertainty, the method discovers and learns the correct region of parameter space, even when initial training bounds exclude the true parameters. This provides a principled, Bayesian justification for adaptive domain augmentation and ensures robust inference for inverse problems under incomplete prior knowledge. We demonstrate the effectiveness of our inverse solver for a toy inverse problem with infinite solutions, and for the parameterization of the quantum correlation functions to event observables in a Quantum Chromodynamics analysis of nucleon structure.
Generating protein sequences that reliably fold into target structures is a central challenge in computational biology and protein design. Progress in protein inverse folding (PIF), however, is fundamentally constrained by the scarcity of high-quality structural data, which limits the effectiveness of existing models. In this work, we propose TTS-Design, a test-time compute scaling framework that enhances protein sequence design without retraining models or relying on larger training data. For TTS-Design, its core design is a dedicated inverse folding reward model named IF-RM, which quantitatively evaluates structure–sequence compatibility and enables effective selection among multiple candidate sequences at inference time. By integrating IF-RM into existing PIF models, TTS-Design leverages additional computation during inference stage to explore and refine candidate solutions. Extensive experiments on the CATH 4.2 and CATH 4.3 demonstrate that TTS-Design consistently improves sequence recovery and structural reliability across different backbone models, without retraining or increasing model size. Case studies on real-world proteins further confirm its practical effectiveness. Overall, our results highlight test-time compute scaling as a promising and effective paradigm for advancing protein inverse folding and other scientific problems under data-limited scenarios.
High dynamic range (HDR) imaging is crucial for capturing real-world lighting conditions. HDR imaging is traditionally achieved either by fusing multiple exposure frames or via inverse tone mapping from a single SDR image. However, the multi-exposure HDR method is prone to motion-induced artefacts and imposes demanding hardware requirements, limiting its practical applicability. Traditional inverse tone-mapping techniques primarily rely on pixel-wise regression methods, which ignore semantic scene contexts and thus frequently introduce halo artefacts and structural distortions. To address this limitation, this paper proposes MLLM-ITM, a novel inverse tone-mapping framework incorporating multimodal large language models (MLLMs). MLLM-ITM utilizes cross-modal features extracted from a frozen MLLM to simultaneously encode visual features and semantic understanding. These features are integrated into a downstream HDR reconstruction backbone through lightweight adapters, enabling content-aware dynamic-range expansion. Furthermore, the decoupled design between MLLM and HDR backbone avoids costly fine-tuning of MLLMs and remains model-agnostic, allowing effortless substitution with emerging multimodal architectures. Extensive experiments on public benchmarks demonstrate that the proposed MLLM-ITM achieves state-of-the-art performance compared with existing inverse tone-mapping methods, highlighting the effectiveness of cross-modal semantic priors in enhancing HDR imaging performance.
Inverse reinforcement learning (IRL) has made significant progress in recovering reward functions from expert demonstrations. However, a key challenge remains: how to extract reward functions that generalize across related but distinct tasks. In this paper, we address this by focusing on transferable IRL, learning intrinsic rewards that can drive effective behavior in unseen but structurally aligned environments. Our method leverages a variational autoencoder to learn an abstract representation of the state space shared across multiple source tasks. This abstracted space captures high-level features that are invariant across tasks, enabling the learning of a unified abstract reward function. The learned reward is then used to train policies in a separate, previously unseen target task without requiring new demonstrations in the target task. We evaluate our approach on multiple environments from Gymnasium and AssistiveGym, demonstrating that the learned abstract rewards consistently support successful policy learning in novel task settings.
Bounded fitting is an attractive paradigm for learning logical formulas from labeled data examples that offers PAC-style generalization guarantees and can often be implemented leveraging SAT solvers. It has been successfully applied to learning concepts of the description logic ALC. We study bounded fitting for learning concepts in expressive description logics that extend ALC with inverse roles, qualified number restrictions, and feature comparisons. We investigate under which conditions bounded fitting keeps its favorable theoretical properties in this setting, and implement is using a SAT solver. We compare our implementation against state-of-the-art concept learners with encouraging results, demonstrating that it is a practical approach to expressive concept learning.
Decision-making pipelines increasingly rely on prediction models whose outputs serve as inputs to downstream optimization problems. Decision-Focused Learning (DFL) has emerged as a promising approach to training such models by directly optimizing decision quality rather than predictive accuracy alone. While most existing DFL methods assume a complete-information setting in which the ground-truth optimization parameters are observed, this paper studies Contextual Inverse Optimization (CIO), an incomplete-information setting in which only the resulting solutions are observed. Prior work on CIO has proposed learning algorithms based on optimality conditions for linear programs, as well as methods that repeatedly solve inverse optimization problems to handle integer programs, often incurring a substantial computational burden. In this paper, we propose a learning algorithm for general optimization problems with linear objective functions that eliminates the need to solve inverse optimization problems. The proposed method learns prediction models by solving a Relaxed Inverse Optimization Problem (RIOP), constructed based on feasible solutions randomly sampled from the feasible region, thereby reducing the computational overhead associated with existing CIO methods. Numerical experiments demonstrate that our method achieves competitive performance in terms of regret compared with existing methods, while offering improved computational efficiency for certain classes of downstream optimization problems.
Many real-world applications require solving families of expensive multi-objective optimization problems~(EMOPs) under varying operational conditions. This can be formulated as parametric expensive multi-objective optimization problems (P-EMOPs) where each task parameter defines a distinct optimization instance. Current multi-objective Bayesian optimization methods have been widely used for finding finite sets of Pareto optimal solutions for each task. However, P-EMOPs present a fundamental challenge: the continuous task parameter space can contain infinite distinct problems, each requiring separate expensive evaluations. To address this, we propose learning an inverse model to amortize the multi-objective optimization cost across the continuous task-preference space, enabling direct solution prediction for any query without the need for expensive re-evaluation. This paper introduces a novel parametric multi-objective Bayesian optimizer that learns this inverse model by alternating between (1) generative solution sampling via conditional generative models and (2) acquisition-driven search leveraging inter-task synergies. This approach enables effective optimization across multiple tasks and finally achieves direct solution prediction for unseen parameterized EMOPs without re-evaluations. We theoretically justify the faster convergence by leveraging inter-task synergies through task-aware Gaussian processes. Based on that, empirical studies in synthetic and real-world benchmarks further verify the effectiveness of the proposed parametric optimizer.
Optimizing survival outcomes, such as patient survival or customer retention, is a critical objective in data-driven decision-making. Off-Policy Evaluation (OPE) provides a powerful framework for assessing such decision-making policies using logged data alone, without the need for costly or risky online experiments in high-stakes applications. However, typical estimators are not designed to handle right-censored survival outcomes, as they ignore unobserved survival times beyond the censoring time, leading to systematic underestimation of the true policy performance. To address this issue, we propose a novel framework for OPE and Off-Policy Learning (OPL) tailored for survival outcomes under censoring. Specifically, we introduce IPCW-IPS and IPCW-DR, which employ the Inverse Probability of Censoring Weighting technique to explicitly deal with censoring bias. We theoretically establish that our estimators are unbiased and that IPCW-DR achieves double robustness, ensuring consistency if either the propensity score or the outcome model is correct. Furthermore, we extend this framework to constrained OPL to optimize policy value under budget constraints. We demonstrate the effectiveness of our proposed methods through simulation studies and illustrate their practical impacts using public real-world data for both evaluation and learning tasks.
Generative models continue to advance rapidly in both fidelity and diversity, posing increasing challenges for reliably distinguishing generated images from real ones. Existing reconstruction-based detection methods often rely on assumptions tied to specific generative models, which leads to limited cross-model generalization and substantial computational overhead. In this paper, we propose General Feature Reconstruction Error (GFRE), a fast and generalizable detection paradigm that leverages reconstruction behavior in a general-purpose representation space. Our key insight is that real and generated images exhibit consistently different reconstruction stability when projected into universal visual representations. Instead of tracing generator-specific artifacts, GFRE employs a lightweight autoencoder to model the reconstructability of image representations, producing a reconstruction signal that is inherently generator-agnostic and transferable across diverse generative processes. Extensive experiments on images synthesized by 18 different generative models demonstrate that GFRE consistently outperforms existing state-of-the-art methods, achieving a 4.70% improvement in detection accuracy and an 8.20% gain in cross-model generalization. Moreover, by avoiding costly generator inversion and diffusion-based reconstruction, GFRE reduces reconstruction error extraction time by up to 150x, enabling efficient and scalable deployment.
Given a social network 𝒢 with n nodes and m edges, the opinion maximization (OM) problem aims at identifying k (k ≪ n) opinion leaders to maximize their opinion propagation in 𝒢. Despite its significance and prevalence, existing OM methods often struggle to strike a satisfactory balance between theoretical performance and practical efficiency. This paper provides an inverse optimization perspective for OM by reformulating it as an opinion minimization (OMin) problem, whose objective function holds succinct expression and desired properties. Then, we introduce a bounded estimation scheme (BES) that can estimate the objective function of OMin with bounded error in Õ(kn+m) expected time, thereby making the optimization for OMin more efficient. To further enhance the optimization efficiency, a decomposition-based decremental estimation algorithm (DDEA) with a near (1-1/e) approximation ratio in Õ(kmn) expected time is specifically designed for OMin. Extensive experiments on real-world social networks of varying scales demonstrate the effectiveness of BES and the superiority of DDEA.
Despite recent advances in training-free text-guided image editing using pretrained rectified flow models, achieving a balance between text adherence and background preservation remains a challenge. Existing methods typically treat image editing as an open-loop trajectory generation task, adopting either an inversion-based or inversion-free paradigm. However, neither can perfectly achieve this goal. Specifically, inversion-based approaches accumulate inversion errors without feedback, whereas inversion-free methods exhibit unstable trajectories due to the lack of constraints. To address this, we propose EKFEdit, a novel framework that formulates image editing as a sequential state estimation problem in a nonlinear dynamic system, which models inversion-free editing trajectories as the system’s prior dynamics and leverages an Extended Kalman Filter to adaptively fuse observations from external editing algorithms. EKFEdit enables adaptive trajectory correction, effectively combining the complementary strengths of the different paradigms. We instantiate this framework into two concrete approaches (EKFEdit-RF and EKFEdit-DNA) via different observation methods, respectively. Extensive experiments demonstrate that our approaches achieve new state-of-the-art performance compared to previous methods. Code is available at https://github.com/baiyeweiguang/EKFEdit.
Gradient inversion attacks (GIAs) have shown that shared gradients in federated learning leak private training data. However, current textual GIAs fail in large batch size, as simply increasing batch size can serve as a stable defense against such attacks. In this paper, we propose CDO-GIA, a novel textual gradient inversion attack via continuous-discrete optimization that extends the maximum effective batch size from 4 to 8 for reconstructing text from gradient. In continuous optimization phase, CDO-GIA incorporates a dynamic weight decay mechanism to alleviate embedding homogenization caused by embedding regularization at large batch sizes, thereby enhancing unigrams reconstruction accuracy. Moreover, our method introduces a tabu beam search mechanism guided by the language model prior during discrete optimization. This design facilitates fine-grained exploration of high-dimensional token spaces via precise token order adjustment, thus reconstructing semantically coherent sequences. By alternating optimization between continuous and discrete phases, CDO-GIA effectively doubling practical attack limit of prior textual GIAs. Extensive experiments are conducted on three binary text classification datasets. The experimental results indicate that CDO-GIA surpasses all baseline methods. With a batch size of 8, it achieves performance gains of 25.01%, 155.64%, and 22.72% over the baselines in terms of Rouge-1, Rouge-2 and Rouge-L scores.
Diffusion models show significant potential for low-light image enhancement. However, this task requires satisfying human perceptual preferences and content fidelity transcending simple brightness and color improvement. Existing methods rely on heuristic physical priors or incorporate perceptual metrics directly into timestep-wise training objectives. Such proxy constraints fail to provide reliable trajectory-level guidance for perceptual alignment. Furthermore, they often lead to artifacts or unnatural visual effects in this ill-posed inverse problem. To address these issues, we propose an unsupervised low-light enhancement framework based on Group Relative Policy Optimization (GRPO), which utilizes perceptual preferences to directly optimize the diffusion policy. We introduce a Sliding Window Hybrid ODE-SDE Sampling strategy that confines stochasticity to dynamic sub-intervals, thereby achieving efficient coarse-to-fine exploration and precise advantage attribution. To meet strict fidelity constraints, we construct a Synergistic Perception-Fidelity Reward and introduce an Independent Advantage Estimation strategy to mitigate signal collapse and gradient suppression in multi-objective optimization. Furthermore, we design a Temporal-Aware Dynamic Weighting Mechanism that adaptively adjusts perceptual weights across denoising stages to balance visual enhancement with structural preservation. Extensive experiments on multiple real-world benchmarks demonstrate that our method effectively improves the perceptual performance of existing models, yielding results that better align with human aesthetics.
In this demo, we present GenMatLab, a user-friendly web platform that makes latest AI techniques accessible for inverse materials design. The platform integrates data analysis and generative modeling into an easy-to-use interface, enabling researchers, material domain experts, and practitioners to explore and apply AI techniques without requiring advanced coding expertise. At its core are generative AI models that support interactive operations, allowing users to conduct inverse design and investigate generated candidates in an intuitive and exploratory way. By lowering technical barriers, GenMatLab empowers a broader community to leverage cutting-edge AI methods for accelerating materials discovery.
Out-of-distribution (OOD) detection and uncertainty estimation (UE) are critical components for building safe machine learning systems. In this work, we propose a novel framework that combines network inversion with classifier training to simultaneously address both OOD detection and uncertainty estimation. We extend a standard n-class classifier by adding an (n+1)-th "garbage" class to capture outliers, initially populated with random Gaussian noise. After each training epoch, we use network inversion to reconstruct inputs for all classes; incoherent reconstructions are assigned to the garbage class for retraining. This iterative cycle of training, inversion, and exclusion continues until inverted samples resemble in-distribution data and uncertainty drops, indicating learned decision boundaries and cleaner class manifolds. At inference, the model detects OOD inputs by classifying them as garbage, with confidence scores estimating uncertainty. Unlike prior methods, this approach requires no external OOD data or post-hoc calibration, providing a simple, unified solution for robust classification, OOD detection, and uncertainty estimation.
Shortcut learning, where machine learning models exploit spurious correlations in data instead of capturing meaningful features, poses a significant challenge to building generalizable models. Vision classifiers based on Convolutional Neural Networks (CNNs), Multi-Layer Perceptrons (MLPs), and Vision Transformers (ViTs) leverage distinct architectural principles to process spatial and structural information, making them differently susceptible to shortcut learning. In this study, we systematically evaluate these architectures by introducing deliberate shortcuts into the dataset that are correlated with class labels both positionally and via intensity, creating a controlled setup to assess whether models rely on these artificial cues or learn actual distinguishing features. We perform both quantitative evaluation by training on the shortcut-modified dataset and testing on two different test sets—one containing the same shortcuts and another without them—to determine the extent of reliance on shortcuts. Additionally, qualitative evaluation is performed using network inversion-based reconstruction techniques to analyze what the models internalize in their weights, aiming to reconstruct the training data as perceived by the classifiers. Further, we evaluate susceptibility to shortcut learning across different learning rates. Our analysis reveals that CNNs at lower learning rates tend to be more reserved against entirely picking up shortcut features, while ViTs, particularly those without positional encodings, almost entirely ignore the distinctive image features in the presence of shortcuts.
Learning More from Less: Resource-Constrained Generative AI for Classification, Generation, and Personalization
PDF ↗The rapid advancement of generative models has created new opportunities for addressing core challenges in computer vision, including data scarcity, image quality, and efficient personalization. My research develops principled, resource- aware methods that enable models to generalize effectively from limited supervision, adapt efficiently to new concepts, and generate high-fidelity visual content. I first address few-shot learning through augmentation-driven uncertainty- guided mixup, improving robustness in data-constrained regimes. Building on this, I propose caption-guided multi-modal augmentation techniques that enrich visual diversity while mitigating real-to-synthetic domain gaps. To enhance the quality and realism of generated images, I introduce diffusion models grounded in natural image statistics, yielding perceptually aligned outputs suitable for downstream tasks. To advance personalization, I develop parameter-efficient mechanisms for combining low-rank adapters, enabling fine-grained control over content and style without retraining. I further extend personalization to a zero-shot setting through a training-free textual-inversion-based method that customizes arbitrary objects directly within the diffusion process. Finally, I present a frequency-guided multi-LoRA fusion framework that leverages wavelet-domain cues and timestep-aware weighting for accurate, training-free concept composition. Collectively, these contributions move toward a unified vision of generative models that are efficient, adaptive, and capable of high-quality, customizable image synthesis.
This paper surveys studies on the use of neural networks for optimization in the training-data-free setting. Specifically, we examine the dataless application of neural network architectures in optimization by re-parameterizing problems using fully connected (or MLP), convolutional, graph, and quadratic neural networks. Although MLPs have been used to solve linear programs a few decades ago, this approach has recently gained increasing attention due to its promising results across diverse applications, including those based on combinatorial optimization, inverse problems, and partial differential equations. The motivation for this setting stems from two key (possibly over-lapping) factors: (i) data-driven learning approaches are still underdeveloped and have yet to demonstrate strong results, as seen in combinatorial optimization, and (ii) the availability of training data is inherently limited, such as in medical image reconstruction and other scientific applications. In this paper, we define the dataless setting and categorize it into two variants based on how a problem instance—defined by a single datum—is encoded onto the neural network: (i) architecture-agnostic methods and (ii) architecture-specific methods. Additionally, we discuss similarities and clarify distinctions between the dataless neural network (dNN) settings and related concepts such as zero-shot learning, one-shot learning, lifting in optimization, and over-parameterization.
Near surface temperature inversions are periods in which a low layer of warm air is trapped between cooler air higher up in the atmosphere and dense cooler air below it near the surface level. By causing cooler air to pool near the surface level, inversions can have detrimental effects for crop growers, including frost, increased moisture, and pesticide drift. As a result, predicting the occurrence and magnitude of these inversions yields substantial benefits for growers. We introduce a Long Short-Term Memory (LSTM) model for temperature inversion forecasting that is able to effectively predict localized, near surface temperature inversions in advance such that growers can take actions to mitigate the detrimental effects. We show a substantial performance gain over a deployed temperature inversion forecasting system, and include a series of ablations that show the benefit of using publicly available terrain-specific feature information when modeling inversions at this scale.