Designing biological sequences such as proteins and DNA for desired properties is challenging due to vast search spaces and limited wet lab evaluation budgets. Current evolutionary approaches ignore sequential dependencies and rely on random mutations, which scale poorly for long sequences. In contrast, reinforcement learning (RL) and generative models explicitly model sequence structure but require large datasets to guide generation toward the target properties. These limitations suggest the need for a method that combines the sample efficiency of evolutionary approaches with the ability to exploit sequential structure. In this work, we propose a novel evolutionary approach, $\texttt{IDEAS}$, in which mutations are guided by an explainable model. The model identifies critical motifs in high-fitness sequences and uses them to mutate non-critical positions. Across six continuous-property datasets, seven baselines, and three evaluation budgets, $\texttt{IDEAS}$ achieves a 19% acceleration in design while maintaining a favorable position on the Pareto curve balancing acceleration, diversity, and novelty.
论文检索
输入标题、作者或关键词，从 13,033 篇学术成果中精准定位
Social Aspects · Accountability, Transparency, and Interpretability
Activation steering offers a computationally efficient mechanism for controlling Large Language Models (LLMs) without fine-tuning. While effectively controlling target traits (e.g., persona), coherency degradation remains a major obstacle to safety and practical deployment. We hypothesize that this degradation stems from intervening on the residual stream, which indiscriminately affects aggregated features and inadvertently amplifies off-target noise. In this work, we identify a sparse subset of attention heads (only three heads) that independently govern persona and style formation, which we term *Style Modulation Heads*. Specifically, these heads can be localized via geometric analysis of internal representations, combining layer-wise cosine similarity and head-wise contribution scores. We demonstrate that intervention targeting only these specific heads achieves robust behavioral control while significantly mitigating the coherency degradation observed in residual stream steering. More broadly, our findings show that precise, component-level localization enables safer and more precise model control.
Social Aspects · Everything Else
This position paper argues that **adopting AI in organizational practice does not guarantee productivity gains, because human and environmental factors critically moderate the relationship between AI deployment and realized productivity improvements**. Following the advent of high-performance generative models, AI use has been rapidly encouraged in some sectors while being restricted in others. Most practitioners assume that AI brings productivity boosts owing to enhanced technical capabilities, but regardless of apparent performance advances in AI technology, human and environmental factors of the organization may substantially attenuate---or even negate---the effective productivity benefits. We identify five key moderating factors: human resource composition, baseline capability of individuals, learning curve of practitioners, incentives for fair use, and flexibility of objectives. Drawing on the partial equilibrium model of Gries and Naudé (2022), we argue that existing economic frameworks may inadvertently overlook these factors. We revise the existing framework to redefine effective organizational determinants and shed light to practical implications including industry and education, responding to alternative views and calling for action of stakeholders.
Applications · Neuroscience, Cognitive Science
Visual prostheses hold great promise for restoring vision in blind individuals. While researchers have successfully utilized M/EEG signals to evoke visual perceptions during the brain decoding stage of visual prostheses, the complementary process of converting images into M/EEG signals in the brain encoding stage remains largely unexplored, hindering the formation of a complete functional pipeline. In this work, we present a novel image-to-brain signal framework that generates M/EEG from images by leveraging the diffusion transformer architecture enhanced with cross-attention mechanisms. Specifically, we employ a diffusion transformer (DiT) architecture based on denoising diffusion implicit models (DDIM) to achieve brain signal generation. To realize the goal of image-to-brain signal conversion, we use cross-attention mechanisms to align brain signal embeddings with CLIP image embeddings. Moreover, we leverage large language models (LLMs) to generate image captions, and concatenate the resulting CLIP text embeddings with CLIP image embeddings to form unified embeddings for cross-attention alignment, enabling our model to capture core semantic information. Furthermore, we introduce a learnable spatio-temporal position encoding that combines brain region embeddings with temporal embeddings to capture both spatial and temporal characteristics of brain signals. We evaluate the framework on two multimodal benchmark datasets (THINGS-EEG2 and THINGS-MEG) and demonstrate that it generates biologically plausible brain signals.
Deep Learning · Theory
Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2× computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.
Reinforcement Learning · Online
Multi-Objective Reinforcement Learning (MORL) extends standard RL by optimizing policies with respect to multiple, often conflicting, objectives. While max-min MORL has emerged as an effective approach for promoting fairness, its applicability remains limited, particularly when constraints must be incorporated. In this paper, we propose a MORL framework that integrates the max-min criterion with explicit constraint satisfaction. We establish a theoretical foundation for the proposed framework and validate the resulting algorithm through convergence analysis and experiments in tabular settings. We further demonstrate the practical relevance of our approach in simulated building thermal control, multi-objective locomotion control, and greenhouse-gas-emission-aware traffic management. Across these domains, our method effectively balances fairness and constraint satisfaction in multi-objective decision-making.
Probabilistic Methods · Everything Else
False discovery rate (FDR) is a cornerstone of modern multiple testing. However, it often fails to guarantee the reliability of ``marginal" discoveries that lie at the boundary of the rejection set, which are often crucial in high-precision applications. While recent works (Soloff et al., 2024; Xiang et al., 2025) introduced the boundary false discovery rate (bFDR) to control the error probability at the marginal discovery, their method relies on restrictive assumptions such as independence or specific prior distributions. In this paper, we first propose $k$-bFDR, a novel generalization that controls the error probability of the $k$ least significant discoveries. We then provide a systematic investigation into the theoretical relationship between $k$-bFDR and existing error metrics. Furthermore, building upon the closure principle, we develop Domino, a unified framework that guarantees $k$-bFDR control under arbitrary dependence, applicable for both p-values and e-values. We prove the theoretical validity of the proposed Domino algorithm and demonstrate through extensive numerical experiments that it consistently achieves rigorous $k$-bFDR control while identifying trustworthy marginal discoveries. Analyses of real data reveal that $k$-bFDR control yields higher-quality rejection sets with greater practical significance.
Deep Learning · Self-Supervised Learning
Joint-Embedding Predictive Architectures (JEPA) learn view-invariant representations and admit projection-based distribution matching for collapse preventions. Existing approaches regularize representations towards isotropic Gaussian distributions, but inherently favor dense representations and fail to capture the key property of sparsity observed in efficient representations. We introduce Rectified Distribution Matching Regularization (RDMReg), a sliced two-sample distribution-matching loss that aligns representations to a Rectified Generalized Gaussian (RGG) distribution. RGG enables explicit control over expected $\ell_p$ norms and induces $\ell_0$ sparsity through rectifications, while preserving maximum entropy up to rescaling under sparsity constraints. Equipping JEPAs with RDMReg yields Rectified LpJEPA, which strictly generalizes prior Gaussian-based JEPAs. Empirically, Rectified LpJEPA learns sparse, non-negative representations with favorable sparsity–performance trade-offs and competitive downstream performance on image classification benchmarks, demonstrating that RDMReg effectively enforces sparsity while preserving task-relevant information.
Applications · Neuroscience, Cognitive Science
Artificial and biological systems may evolve similar computational solutions despite fundamental differences in architecture and learning mechanisms—a form of convergent evolution. We provide large-scale evidence for this phenomenon through comprehensive analysis of alignment between human brain activity and internal representations across over 600 AI models spanning language and vision domains (1.33M to 72B parameters). Analysis of 60 million alignment measurements reveals that higher-performing models spontaneously develop stronger brain correspondence without explicit neural constraints, with language models demonstrating markedly stronger correlations ($r=0.89, p<7.5 \times 10^{-13}$) than vision models ($r=0.53, p<2.0 \times 10^{-44}$). Crucially, longitudinal training analysis shows that brain alignment consistently emerges prior to performance improvements, suggesting that developing brain-like representations constitutes a fundamental stepping stone toward enhanced capabilities. We identify systematic organizational patterns reflecting human cognitive architecture: language models exhibit strongest alignment with limbic and integrative regions, while vision models show progressive correspondence with visual cortical hierarchies. These findings establish that optimization for task performance naturally drives AI systems toward human-like computational strategies.
Applications · Robotics
Multimodal Large Language Models (MLLMs) have recently emerged as a promising paradigm for vehicle-to-vehicle (V2V) cooperative autonomous driving, enabling language-based joint perception, prediction, and decision-making in safety-critical scenarios with severe occlusions. However, existing V2V–MLLM frameworks rely on dense token-level sharing and fusion, leading to high communication and inference costs. Moreover, conventional V2V perception methods are limited to feature-sharing paradigms without language reasoning, and existing generic token pruning strategies fail to consider LiDAR-specific spatial structure and multi-agent fusion. To address these limitations, we propose V2V Communication-Conditioned MLLM Framework (V2V-CCM), a dual-stage communication coop- erative framework that broadcasts request messages to all agents and uses them to identify redundant visual tokens. Specifically, Question Semantic Message (QSM) encodes the global question intent to guide question-relevant token selection, while Question Semantic Message (QSM) summarizes LiDAR features to identify spatially redundant tokens that are already observed and therefore need not be transmitted. By integrating this strategy into dual-stage frameworks, our method substantially reduces communication and inference costs while preserving question-relevant tokens and spatially redundant tokens. Extensive experiments on the V2V-QA and V2V-GoT-QA datasets demonstrate that V2V-CCM consistently outperforms existing pruning methods and achieves state-of-the-art performance.
Applications · Time Series
Irregular multivariate time series (IMTS) forecasting is challenging due to non-uniform sampling and variable asynchronicity. These irregularities violate the equidistant assumptions of standard models, hindering local temporal modeling and rendering classical frequency-domain methods ineffective for capturing global periodic structures. To address this challenge, we propose TFMixer, a joint time–frequency modeling framework for IMTS forecasting. Specifically, TFMixer incorporates a Global Frequency Module that employs a learnable Non-Uniform Discrete Fourier Transform (NUDFT) to directly extract spectral representations from irregular timestamps. In parallel, the Local Time Module introduces a query-based patch attention mechanism to adaptively aggregate informative temporal segments and alleviate information density imbalance. Finally, TFMixer fuses the time-domain and frequency-domain representations to generate forecasts and further leverages inverse NUDFT for explicit seasonal extrapolation. Extensive experiments on real-world IMTS benchmarks demonstrate the effectiveness and robustness of TFMixer under irregular sampling and missing data.
General Machine Learning · Unsupervised and Semi-supervised Learning
Single-Positive Multi-Label Learning (SPML) studies learning from incomplete supervision, where each instance is annotated with only one positive label despite potentially belonging to multiple categories. While existing methods assume the annotated labels are randomly distributed, real-world annotations are often biased toward the most salient category. We formalize this realistic scenario as Salient Single-Positive Multi-Label Learning (SalSPML). This salient annotation bias poses a challenge to conventional SPML methods, as the missing labels often correspond to less salient and harder-to-recognize categories. Fortunately, we find that salient annotations are typically more representative and informative. Motivated by this insight, we propose Prototype-Guided Rejection for Salient Annotation (PiSA), which constructs reliable class-wise prototypes from salient labels and leverages them to guide embedding learning for non-salient labels recognition. We theoretically demonstrate that SalSPML is harder than Random SPML due to irreducible annotation bias, and under SalSPML, more accurate prototypes facilitate false-negative label detection. Experiments on multiple benchmarks, together with two newly constructed real-world SalSPML datasets, demonstrate that PiSA consistently outperforms existing methods, achieving an average mAP improvement of 3.16\%. Our code is available in the supplementary materials.
We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such as ELU, LeakyReLU, LogSigmoid, PReLU, ReLU, SELU, CELU, Sigmoid, SiLU, Mish, Softplus, Tanh, Softmin, Softmax, and LogSoftmax. For an error target of $\varepsilon>0$, we establish approximation-theoretic separations: Any network built from standard fixed activations can be uniformly approximated on compact domains by a rational-activation network with only $\mathrm{poly}(\log\log(1/\varepsilon))$ overhead in size, while the converse provably requires $\Omega(\log(1/\varepsilon))$ parameters in the worst case. This exponential gap persists at the level of full networks and extends to gated activations and transformer-style nonlinearities. In practice, rational activations integrate seamlessly into standard architectures and training pipelines, allowing rationals to match or outperform fixed activations under identical architectures and optimizers.
General Machine Learning · Transfer, Multitask and Meta-learning
Bayesian predictive inference provides a principled framework for uncertainty quantification, data efficiency, and robust generalization. However, exact inference is often intractable, and scalable approximations may remain computationally expensive or require restrictive modeling assumptions that degrade predictive performance. Prior-Data Fitted and in-context learning networks have recently emerged as an amortized alternative by learning to map datasets directly to predictive distributions, but existing approaches are tightly coupled to the support of the training prior and lack explicit mechanisms for adapting to new priors at test time, resulting in limited robustness under distribution shift. We introduce a multi-task in-context learning framework for amortized hierarchical Bayesian predictive inference that explicitly represents prior information as a prefix of in-context datasets. A transformer trained on sequences of prior and target tasks learns to adapt its predictions across families of priors. On a suite of evaluations with increasing difficulty, including out-of-meta-distribution heavy-tailed priors and priors with high-dimensional latent structures, our method matches oracle Bayesian predictors while being orders of magnitude faster.
Applications · Chemistry, Physics, and Earth Sciences
Machine learning interatomic potentials (MLIPs) achieve excellent accuracy when trained on large Density Functional Theory (DFT) data. To be useful in practice, they must often be adapted to target chemistries using small and expensive task-specific datasets. However, MLIPs transfer inconsistently across domains, with representations that often loose accessible composition and structure information. To address this, we present TriForces, a model-agnostic three-stream framework that separates composition and structure information, combined with self-supervised learning to preserve transferable representations. TriForces improves performance on MatBench and QM9 over baselines without needing DFT labels and enables efficient similar structure retrieval through its learned latent space. On OMat24, in limited-data training regime, TriForces reduces energy MAE by 57\% at 20K samples only and improves force MAE across sample sizes. We release pretrained TriForces variants across multiple MLIP architectures with code at https://anonymous.4open.science/r/triforces-063E.
General Machine Learning · Evaluation
Generative model evaluation commonly relies on high-dimensional embedding spaces to compute distances between samples. We show that dataset representations in these spaces are affected by the hubness phenomenon, which distorts nearest neighbor relationships and biases distance-based metrics. Building on the classical Iterative Contextual Dissimilarity Measure (ICDM), we introduce Generative ICDM (GICDM), a method to correct neighborhood estimation for both real and generated data. We introduce a multi-scale extension to improve empirical behavior. Extensive experiments on synthetic and real benchmarks demonstrate that GICDM resolves hubness-induced failures, restores reliable metric behavior, and improves alignment with human judgment.
General Machine Learning · Transfer, Multitask and Meta-learning
Current meta-learning methods are constrained to narrow task distributions with fixed feature and label spaces, limiting applicability. Moreover, the current meta-learning literature uses key terms like “universal” and “general-purpose” inconsistently and lacks precise definitions, hindering comparability. We introduce a theoretical framework for meta-learning which formally defines practical universality and introduces a distinction between algorithm-explicit and algorithm-implicit learning, providing a principled vocabulary for reasoning about universal meta-learning methods. Guided by this framework, we present TAIL, a transformer-based algorithm-implicit meta-learner that functions across tasks with varying domains, modalities, and label configurations. TAIL features three innovations over prior transformer-based meta-learners: random projections for cross-modal feature encoding, random injection label embeddings that extrapolate to larger label spaces, and efficient inline query processing. TAIL achieves state-of-the-art performance on standard few-shot benchmarks while generalizing to unseen domains. Unlike other meta-learning methods, it also generalizes to unseen modalities, solving text classification tasks despite training exclusively on images, handles tasks with up to 20× more classes than seen during training, and provides orders-of-magnitude computational savings over prior transformer-based approaches.
Applications · Time Series
Time series forecasting is essential in various domains. Compared to relying solely on endogenous variables (i.e., target variables), considering exogenous variables (i.e., covariates) provides additional predictive information and often leads to more accurate predictions. However, existing methods for time series forecasting with exogenous variables (TSF-X) have the following shortcomings: 1) they do not leverage future exogenous variables, 2) they fail to fully account for the correlation between endogenous and exogenous variables. In this study, to better leverage exogenous variables, especially future exogenous variables, we propose $\textbf{DAG}$, which $\textit{utilizes $\underline{D}$ual correl$\underline{A}$tion network along both the temporal and channel dimensions for time series forecasting with exo$\underline{G}$enous}$ variables. Specifically, we propose two core components: the Temporal Correlation Module and the Channel Correlation Module. Both modules consist of a correlation discovery submodule and a correlation injection submodule. The former is designed to capture the correlation effects of historical exogenous variables on future exogenous variables and on historical endogenous variables, respectively. The latter injects the discovered correlation relationships into the processes of forecasting future endogenous variables based on historical endogenous variables and future exogenous variables.
Applications · Time Series
Time series forecasting is important in many fields that require accurate predictions for decision-making. Patching techniques, commonly used and effective in time series modeling, help capture temporal dependencies by dividing the data into patches. However, existing patch-based methods fail to dynamically select patches and typically use all patches during the prediction process. In real-world time series, there are often low-quality issues during data collection, such as missing values, distribution shifts, anomalies and white noise, which may cause some patches to contain low-quality information, negatively impacting the prediction results. To address this issue, this study proposes a robust time series forecasting framework called $\textbf{SEER}$. Firstly, we propose an $\textit{Augmented Embedding Module}$, which improves patch-wise representations using a Mixture-of-Experts~(MoE) architecture and obtains series-wise token representations through a channel-adaptive perception mechanism. Secondly, we introduce a $\textit{Learnable Patch Replacement Module}$, which enhances forecasting robustness and model accuracy through a two-stage process: 1) a dynamic filtering mechanism eliminates negative patch-wise tokens; 2) a replaced attention module substitutes the identified low-quality patches with global series-wise token, further refining their representations through a causal attention mechanism. Comprehensive experimental results demonstrate the SOTA performance of SEER.
Deep Learning · Large Language Models
Despite the success of parameter-efficient fine-tuning (PEFT) methods in reducing parameter-related overhead, fine-tuning large language models (LLMs) is still bottlenecked by significant memory and computational demands. In this paper, we propose **TokenDrop**, a token-level importance-aware backpropagation skipping method that reduces activation memory and accelerates LLM fine-tuning by skipping backward computations for less informative tokens. TokenDrop evaluates token importance based on the magnitude of residual updates during the forward pass, enabling lightweight, gradient-free importance estimation. Furthermore, we introduce cumulative token selection to preserve gradient continuity across layers and lazy selection scheduling that defers token selection to facilitate globally informed importance scoring under memory constraints. Across a range of experiments, TokenDrop achieves up to **42.9**\% reduction in memory usage and up to **1.50**$\times$ training speedup, while preserving accuracy and outperforming existing backpropagation-skipping baselines. The code is available at https://anonymous.4open.science/r/tokendrop_official-B469.