Bound by semanticity: universal laws governing the generalization-identification tradeoff
Polytechnic University of Turin · Northeastern University · Northeastern University London · Intesa Sanpaolo AI Research · Princeton University · Dartmouth College · Carnegie Mellon University · University of California, Los Angeles · Politecnico di Torino · Network Science Institute, Northeastern University London
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Intelligent systems must form internal representations that support both broad generalization and precise identification. Here, we show that these two goals are fundamentally in tension with one another. We derive closed-form expressions proving that any model whose representations have a finite semantic resolution, impairing long-range similarity computations, must lie on a universal Pareto front linking its probability of correct generalization $p_S$ and identification $p_I$. We extend this analysis to general input spaces and to parallel processing scenarios, predicting a sharp $1/n$ collapse in the capacity of processing multiple inputs at the same time. A minimal ReLU network reproduces these laws: a resolution boundary emerges during learning, and empirical $(p_S,p_I)$ trajectories closely match the theory for linearly decaying similarity. Finally, we show that the same limits appear in far more complex systems, including a convolutional neural network and state-of-the-art vision–language models, indicating that learned finite-resolution similarity are broad and foundational informational constraints rather than toy-model artifacts. Together, these results provide a precise theory of the generalization–identification tradeoff and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike.