Activation Shape Matters: OOD Detection with Norm-Entropy Fusion

摘要

Out-of-distribution (OOD) detection is crucial for safe ML deployment, yet neural networks often exhibit overconfidence on unseen data. While activation norms provide useful OOD signals, they remain vulnerable---OOD inputs can artificially inflate norms through sparse, high-magnitude activations, while valid in-distribution samples with moderate norms may be misclassified. We propose that activation distributional shape, not just magnitude, is essential for robust detection. Our method, Activation Norm and Entropy Weighting (ANEW), combines L2-norm (strength) with Shannon entropy (spread) to distinguish between genuine in-distribution patterns and OOD samples, including adversarial examples mimicking high norms via low-entropy spikes. ANEW requires only a single forward pass without retraining, making it highly practical. Extensive experiments across diverse architectures and benchmarks show ANEW significantly outperforms norm-only baselines, reducing both false positives and false negatives in challenging scenarios. Code available upon acceptance.