Symbolic Regression (SR) is the task of finding a closed-form mathematical expression that optimizes some objective. Solving this task is NP-hard. However, SR software routinely discovers accurate, interpretable models without exhaustively searching function space. Motivated by this disconnect between worst-case theory and practical success, we study SR through the lens of *parameterized complexity theory*. In particular, we reanalyze tractability with respect to practically relevant parameters like expression depth, tree size, and number of primitives used. We show that SR is actually fixed-parameter tractable (FPT) under a parametrization over expression depth or tree size, formalizing an explanation for why the bounded-complexity search of popular SR algorithms succeeds. However, SR becomes W[1]-hard when parameterized by the number of variables or primitives used, identifying selection as a source of intractability. We further find lower bounds under the exponential time hypothesis, prove approximation hardness, and rule out polynomial kernels.
论文检索
输入标题、作者或关键词，从 13,033 篇学术成果中精准定位
Deep Learning · Generative Models and Autoencoders
Diffusion models serve as generative priors for dataset distillation, yet existing pipelines rely on per-sample update rules that evolve each synthetic image independently, limiting their ability to optimize collective set-level objectives. We propose Set-Coupled Guidance (SCG), a plug-and-play auxiliary controller that shifts from per-image to group (IPC-at-once) sampling by injecting set-symmetric feedback at each diffusion step. SCG combines spectral set-point regulation, which aligns set-level statistics to real data via empirical characteristic function matching, with cooperative kernel coupling that stabilizes joint trajectories under noisy feedback. All computations operate on lightweight descriptors extracted from predicted clean latents, adding low overhead to the base method. We provide theoretical analysis including Lyapunov descent and input-to-state stability for distributional tracking. Experiments on ImageNette, ImageWoof, ImageNet-100 and ImageNet-1K show consistent accuracy gains across multiple diffusion-based baselines.
Probabilistic Methods · Everything Else
Dominant approaches for modelling Partial Differential Equations (PDEs) rely on deterministic predictions, yet many physical systems of interest are inherently chaotic and uncertain. While training probabilistic models from scratch is possible, it is computationally expensive and fails to leverage the significant resources already invested in high-performing deterministic backbones. In this work, we adopt a training-efficient strategy to transform pre-trained deterministic models into probabilistic ones via retrofitting with a proper scoring rule: the Continuous Ranked Probability Score (CRPS). Crucially, this approach is architecture-agnostic: it applies the same adaptation mechanism across distinct model backbones with minimal code modifications. The method proves highly effective across different scales of pre-training: for models trained on single dynamical systems, we achieve $20–54\%$ reductions in rollout CRPS and up to $30\%$ improvements in variance-normalised RMSE (VRMSE) relative to compute-matched deterministic fine-tuning. We further validate our approach on a PDE foundation model, trained on multiple systems and retrofitted on the dataset of interest, to show that our probabilistic adaptation yields an improvement of up to $40\%$ in CRPS and up to $15\%$ in VRMSE compared to deterministic fine-tuning. Validated across diverse architectures and dynamics, our results show that probabilistic PDE modelling need not require retraining from scratch, but can be unlocked from existing deterministic backbones with modest additional training cost.
Applications · Chemistry, Physics, and Earth Sciences
We present protein autoregressive modeling (PAR), the first multi-scale autoregressive framework for protein backbone generation via coarse-to-fine next-scale prediction. Using the hierarchical nature of proteins, PAR generates structures that mimic sculpting a statue, forming a coarse topology and refining structural details over scales. To achieve this, PAR consists of three key components: (i) multi-scale downsampling operations that represent protein structures across multiple scales during training; (ii) an autoregressive transformer that encodes multi-scale information and produces conditional embeddings to guide structure generation; (iii) a flow-based backbone decoder that generates backbone atoms conditioned on these embeddings. Moreover, autoregressive models suffer from exposure bias, caused by the training and the generation procedure mismatch, and substantially degrades structure generation quality. We effectively alleviate this issue by adopting noisy context learning and scheduled sampling, enabling robust backbone generation. Notably, PAR exhibits strong zero-shot generalization, supporting flexible human-prompted conditional generation and motif scaffolding without requiring fine-tuning. On the unconditional generation benchmark, PAR effectively learns protein distributions and produces backbones of high design quality, and exhibits favorable scaling behavior. Together, these properties establish PAR as a promising framework for protein structure generation.
Applications · Chemistry, Physics, and Earth Sciences
We present protein autoregressive modeling (PAR), the first multi-scale autoregressive framework for protein backbone generation via coarse-to-fine next-scale prediction. Using the hierarchical nature of proteins, PAR generates structures that mimic sculpting a statue, forming a coarse topology and refining structural details over scales. To achieve this, PAR consists of three key components: (i) multi-scale downsampling operations that represent protein structures across multiple scales during training; (ii) an autoregressive transformer that encodes multi-scale information and produces conditional embeddings to guide structure generation; (iii) a flow-based backbone decoder that generates backbone atoms conditioned on these embeddings. Moreover, autoregressive models suffer from exposure bias, caused by the training and the generation procedure mismatch, and substantially degrades structure generation quality. We effectively alleviate this issue by adopting noisy context learning and scheduled sampling, enabling robust backbone generation. Notably, PAR exhibits strong zero-shot generalization, supporting flexible human-prompted conditional generation and motif scaffolding without requiring fine-tuning. On the unconditional generation benchmark, PAR effectively learns protein distributions and produces backbones of high design quality, and exhibits favorable scaling behavior. Together, these properties establish PAR as a promising framework for protein structure generation.
Controlling generative models—whether via inference-time steering or fine-tuning—is expensive. Control relies on estimating the value function—typically necessitating costly trajectory simulations. To eliminate this bottleneck, we introduce *Meta Flow Maps (MFMs)*, stochastic extensions of consistency models and flow maps. MFMs are trained to perform \textbf{one-step posterior sampling}, generating arbitrarily many i.i.d. draws of clean data $x_1$ from any noisy state $x_t$. Crucially, these samples are differentiable in the conditioning state $x_t$, unlocking efficient estimation of the value function gradient. We leverage this capability to enable both **inference-time steering** without inner rollouts, and unbiased, off-policy **fine-tuning** to general rewards. Among our fine-tuning and steering experiments on ImageNet, we highlight that our single-particle steered-MFM sampler outperforms a Best-of-1000 baseline across multiple rewards at a fraction of the compute.
Social Aspects · Alignment
Ensuring the safety of Generative AI requires a nuanced understanding of pluralistic viewpoints. In this paper, we introduce a novel data-driven approach for analyzing ordinal safety ratings in pluralistic settings. Specifically, we address the challenge of interpreting nuanced differences in safety feedback from a diverse population expressed via ordinal scales (e.g., a Likert scale). We define non-parametric responsiveness metrics that quantify how raters convey broader distinctions and granular variations in the severity of safety violations. Leveraging publicly available datasets of pluralistic safety feedback as our case studies, we investigate how raters from different demographic groups use an ordinal scale to express their perceptions of the severity of violations. We apply our metrics across violation types, demonstrating their utility in extracting nuanced insights that are crucial for aligning AI systems reliably in multi-cultural contexts. We show that our approach can inform rater selection and feedback interpretation by capturing nuanced viewpoints across different demographic groups, hence improving the quality of pluralistic data collection and in turn contributing to more robust AI alignment.
Social Aspects · Alignment
This position paper argues that modern alignment methods – originally designed to prevent harmful output – are dual-use technologies that may easily be misused by malicious actors for censorship and manipulation. By mapping current alignment techniques to the possibility and actual cases of misuse, we show that the quest for a ''perfectly aligned'' model inadvertently also provides malicious actors with an ever-improving tool for informational dominance. We need to discuss this dual-use potential *now*, as its risk is exacerbated by rapid user adoption of AI as information provider and a political landscape that increasingly shifts towards authoritarianism. We conclude by urging the community to consider the intentional misuse of safety mechanisms and propose mitigation strategies to safeguard against this dual-use potential.
General Machine Learning · Causality
The problem of domain generalization concerns learning predictive models that are robust to distribution shifts when deployed in new, previously unseen environments. Existing methods typically require labeled data from multiple training environments, limiting their applicability when labeled data are scarce. In this work, we study domain generalization in an anti-causal setting, where the outcome causes the observed covariates. Under this structure, environment perturbations that affect the covariates do not propagate to the outcome, which motivates regularizing the model's sensitivity to these perturbations. Crucially, estimating these perturbation directions does not require labels, enabling us to leverage unlabeled data from multiple environments. We propose two methods that penalize the model's sensitivity to variations in the mean and covariance of the covariates across environments, respectively, and prove that these methods have worst-case optimality guarantees under certain classes of environments. Finally, we demonstrate the empirical performance of our approach on a controlled physical system and a physiological signal dataset.
Deep Learning · Large Language Models
Chain-of-Thought (CoT) reasoning has significantly improved the performance of Large Language Models (LLMs) but comes with substantial computational costs due to excessive token consumption. Existing approaches to reduce inference latency, such as explicit length penalties, often degrade reasoning quality by truncating necessary logical steps. In this work, we introduce a novel, SFT-free reinforcement learning framework that induces emergent token efficiency without explicit length constraints.We propose Batched Contextual Reinforcement (BCR), a training paradigm where the model is prompted to solve multiple reasoning tasks within a single context window, rewarded by independent instance-level accuracy. This formulation introduces an implicit information bottleneck: to maximize the cumulative reward within the context capacity, the model is forced to eliminate syntactic redundancy and focus attention on the semantic core of the reasoning path.Empirically, our method demonstrates a remarkable shift in the efficiency-accuracy Pareto frontier. Using a 1.5B parameter model JustRL-Deepseek-1.5B, we achieve **39.8--62.6% reduction in token usage** across five mathematical reasoning benchmarks while maintaining or improving accuracy on four of them. Most notably, on AMC23 and Minerva, we observe a ``free lunch'' phenomenon where accuracy improves by **+2.5%** and **+5.1%** respectively, despite using approximately **half the tokens**. Extensive ablation studies confirm that batched training acts as a superior form of implicit regularization that reduces hallucinations and sharpens attention. Our findings indicate that LLMs possess latent, high-density reasoning modes that can be unlocked via purely structural incentives in RL.
Applications · Health / Medicine
Medical diagnosis demands models that can process multimodal medical inputs, such as medical images and patient histories, and generate diverse outputs including textual reports and visual content, such as annotations or segmentation masks. Despite this need, existing medical AI models disrupt this unified process: image understanding models interpret images without producing visual outputs, while image generation models produce visual outputs but cannot provide textual explanations. Therefore, we propose a multi-level framework called Observation-Knowledge-Analysis (OKA) to unify the distinct processes. Specifically, at the observation level, we construct UniMed-5M, a dataset comprising over 5.6M samples that reformat diverse unimodal data into multimodal pairs. At the knowledge level, we propose Progressive Curriculum Learning, where models simultaneously learn medical multimodal understanding and generation knowledge from UniMed-5M.At the analysis level, we introduce UniMedVL, the first medical unified multimodal model that unifies image understanding and generation within a single architecture without manually reloading model checkpoints. UniMedVL achieves superior performance on 5 medical image understanding benchmarks, while matching specialized models in generation quality across 8 medical imaging modalities. Crucially, our unified architecture enables bidirectional knowledge sharing, improving performance on both image understanding and generation tasks. Code is available at https://anonymous.4open.science/r/Uni-MedVL-65F2/README.md.
Applications · Health / Medicine
The longitudinal management of blinding fundus diseases constitutes a Partially Observable Markov Decision Process (POMDP) necessitating a critical precision-risk trade-off between intervention and over-treatment, as true pathology is often obscured in static observations. However, existing paradigms fail to address this complexity. Traditional vision models remain uninterpretable and memoryless, and while Vision-Language Models (VLMs) excel in semantic understanding, they rely on unsafe open-loop text reasoning lacking the anatomical grounding essential for clinical safety. Furthermore, robust learning is hindered by the scarcity of process supervision in sparse clinical records. To bridge this gap, we introduce the {Logic-Constrained Abductive Data Engine}. Operating on a ``Propose-and-Verify'' paradigm, it validates MLLM-Proposed biomarkers against clinical and temporal logic to reconstruct dense pathological states from sparse outcomes. Building on this foundation, we propose ORBIT, the first ophthalmic Prognostic World Model. Uniquely, ORBIT employs counterfactual visual foresight to imagine anatomical futures under different treatments, anchoring decisions in Closed-Loop Anatomical Verification rather than linguistic probabilities. Experiments demonstrate that ORBIT effectively captures disease evolution and establishes a new paradigm for autonomous diagnosis and reliable decision-making in complex ophthalmic environments.
Applications · Computer Vision
Cross-modal 2D–3D gait recognition is impeded by inherent domain discrepancies between 2D silhouette and 3D point cloud distributions. While prior methods align only final embeddings, we propose DiffCrossGait, which enforces trajectory-level alignment by driving both modalities with shared noise in a unified latent diffusion process. By driving both modalities with shared Gaussian noise within a latent space, we enable continuous alignment throughout the generative evolution. We introduce a Tri-Phase Alignment Strategy that exploits varying noise intensities to enforce identity anchoring, dynamics consistency, and cross-modal structural recoverability, thereby constraining both modalities to share denoising dynamics and bottleneck structure, which promotes modality-invariant gait features. Crucially, our framework decouples generative alignment from the discriminative backbone; the diffusion mechanism serves exclusively as a training objective, ensuring high inference efficiency by eliminating the computational overhead of iterative denoising. Extensive experiments on the SUSTech1K and FreeGait benchmarks demonstrate that DiffCrossGait achieves state-of-the-art performance.
Optimization · Large Scale, Parallel and Distributed
We study the role of batch size in stochastic conditional gradient methods under a $\mu$-Kurdyka–Łojasiewicz ($\mu$-KL) condition. Focusing on momentum-based stochastic Frank–Wolfe–type conditional gradient algorithms (e.g., Scion), we derive a new analysis that explicitly captures the interaction between stepsize, batch size, and stochastic noise. Our study reveals a regime-dependent behavior: increasing the batch size initially improves optimization accuracy, but beyond a critical threshold, the benefits saturate and can eventually degrade performance under a fixed token budget. Notably, the theory predicts the magnitude of the optimal stepsize and aligns well with empirical practices observed in large-scale training. Leveraging these insights, we derive principled guidelines for selecting the batch size and stepsize, and propose an adaptive strategy that increases batch size and sequence length during training while preserving convergence guarantees. Preliminary experiments are consistent with the theoretical predictions and illustrate the emergence of the predicted scaling regimes. Overall, our results provide a theoretical framework for understanding batch-size scaling in stochastic conditional gradient methods and offer guidance for designing efficient training schedules in large-scale optimization.
Social Aspects · Accountability, Transparency, and Interpretability
This paper concerns the question of how language models and other AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation of this paper is that the natural geometry of these representation spaces should reflect the way models use representations to produce behavior. We focus on the important special case of representations that define softmax distributions. We argue that the natural geometry is information geometry, and then show how this interacts with semantic encoding and the linear representation hypothesis. It turns out that the duality structure of information geometry plays a critical role. As an illustrative application, we develop *dual steering*, a method for robustly steering representations to exhibit a particular concept using linear probes. We formally prove that dual steering optimally modifies the target concept while minimizing changes to off-target concepts. We empirically find that dual steering enhances the controllability and stability of concept manipulation.
Reinforcement Learning · Multi-agent
In this work, we present the first theoretical analysis of multi-agent imitation learning (MAIL) in linear Markov games where both the transition dynamics and each agent's reward function are linear in some given features. We demonstrate that by leveraging this structure, it is possible to replace the state-action level \emph{all policy deviation concentrability coefficient} \citep{freihaut2025rate} with a concentrability coefficient defined at the feature level which can be much smaller than the state-action analog when the features are informative about \emph{states' similarity}. Furthermore, to circumvent the need for any concentrability coefficient, we turn to the interactive setting. We provide the first, computationally efficient, interactive MAIL algorithm for linear Markov games and show that its sample complexity depends only on the dimension of the feature map $d$. Building on these theoretical findings, we propose a deep MAIL interactive algorithm which clearly outperforms BC on games such as Tic-Tac-Toe and Connect4.
Applications · Time Series
Event sequences from complex systems, such as clinical workflows, are often sparse and incomplete. As a result, downstream models are trained on data that only partially captures the underlying dynamics. Synthetic sequence generation can augment real data by filling in missing structure and improving coverage of rare patterns, but generated trajectories must remain realistic, satisfy domain constraints, and allow control. We propose the Forward-Chaining Temporal Point Process (FC-TPP), a framework for constraint-aware and controllable sequence generation in continuous time. FC-TPP maintains an explicit latent symbolic state encoding high-level predicates, which evolves through a differentiable multi-hop forward-chaining operator. Logical rules update the latent state based on recent events, while a temporal point process decoder generates future event times and types conditioned on this evolving state. By tying the generative dynamics to multi-hop reasoning in latent space, FC-TPP incorporates symbolic structure throughout generation rather than relying directly on raw event histories. Experiments on synthetic data and four semi-synthetic/real-world benchmarks—LogiCity, MIMIC-IV, EPIC-100, and IKEA ASM—show that FC-TPP achieves higher generation quality under limited and incomplete data, with stronger constraint adherence and greater controllability than purely neural and prior neuro-symbolic baselines.
Deep Learning · Everything Else
Deep learning models for supervised learning on tabular data are rapidly improving. Notably, ensembles (mixtures of multiple models) often play an important role in achieving top performance, which motivates designing ensemble-first systems rather than treating ensembling as an ad hoc trick. In this work, we present TabPack --- a new ensembling approach that packs many base model-optimizer pairs with different hyperparameters into a single neural network and a single optimizer. The base model-optimizer hyperparameters are sampled randomly, after which all base models are trained in parallel, and the final ensemble is built on the fly during training. As a result, TabPack produces powerful ensembles in a single run, with substantial efficiency gains over traditional approaches. With its remarkable efficiency, strong performance on public benchmarks, and reduced reliance on traditional hyperparameter tuning, TabPack becomes an appealing solution for practitioners, and suggests a new avenue for designing better tabular deep learning systems.
General Machine Learning · Representation Learning
Smoothing a signal based on local neighborhoods is a core operation in machine learning and geometry processing. On well-structured domains such as vector spaces and manifolds, the Laplace operator derived from differential geometry offers a principled approach to smoothing via heat diffusion, with strong theoretical guarantees. However, constructing such Laplacians requires a carefully defined domain structure, which is not always available. Most practitioners thus rely on simple convolution kernels and message-passing layers, which are biased against the boundaries of the domain. We bridge this gap by introducing a broad class of smoothing operators, derived from general similarity or adjacency matrices, and demonstrate that they can be normalized into diffusion-like operators that inherit desirable properties from Laplacians. Our approach relies on a symmetric variant of the Sinkhorn algorithm, which rescales positive smoothing operators to match the structural behavior of heat diffusion. This construction enables Laplacian-like smoothing and processing of irregular data such as point clouds, sparse voxel grids or mixture of Gaussians. We show that the resulting operators not only approximate heat diffusion but also retain spectral information from the Laplacian itself, with applications to shape analysis and matching.
Deep Learning · Large Language Models
Large language models (LLMs) are increasingly applied as automatic evaluators for natural language generation assessment often using pairwise comparative judgements. Existing approaches typically rely on single judges or aggregate multiple judges assuming equal reliability. In practice, LLM judges vary substantially in performance across tasks and aspects, and their judgment probabilities may be biased and inconsistent. Furthermore, human-labelled supervision for judge calibration may be unavailable. We first empirically demonstrate that inconsistencies in LLM comparison probabilities exist and show that it limits the effectiveness of direct probability-based ranking. To address this, we study the \emph{LLM-as-a-jury} setting and propose BT-$\sigma$, a judge-aware extension of the Bradley–Terry model that introduces a discriminator parameter for each judge to jointly infer item rankings and judge reliability from pairwise comparisons alone. Experiments on benchmark NLG evaluation datasets show that \textit{BT-$\sigma$} consistently outperforms averaging-based aggregation methods, and that the learned discriminator strongly correlates with independent measures of LLM evaluation performance. Further analysis reveals that \textit{BT-$\sigma$} can be interpreted as an unsupervised calibration mechanism that improves aggregation by modelling judge reliability.