dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning
Datacurve · University of Toronto, Arc Institute · University of Pennsylvania, University of Pennsylvania · NVIDIA, Arc Institute, UToronto, Stanford University · onepot · CMU · Vector Institute · University of California, Berkeley · Arc Institute · Cartesia AI CMU
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Genomic foundation models have the potential to decode DNA syntax, yet face a fundamental tradeoff. Standard subword tokenizers fragment biologically meaningful motifs such as codons and regulatory elements, while nucleotide-level models preserve biological coherence but incur prohibitive computational costs for long contexts. We introduce dnaHNet, a state-of-the-art tokenizer-free autoregressive model that segments and models genomic sequences end to end. Using a differentiable dynamic chunking mechanism, dnaHNet compresses raw nucleotides into latent tokens adaptively, balancing compression with predictive accuracy. Pretrained on prokaryotic genomes, dnaHNet outperforms leading architectures including StripedHyena2 in scaling and efficiency. This recursive chunking yields quadratic FLOP reductions, enabling $>3 \times$ inference speedup over Transformers. On zero-shot tasks, dnaHNet achieves superior performance in predicting protein variant fitness and gene essentiality, while automatically discovering hierarchical biological structures without supervision. These results establish dnaHNet as a scalable, interpretable framework for next-generation genomic modeling.