EBaR: Efficient Buffer and Resetting for Single-Sample Continual Test-Time Adaptation

摘要

In test-time adaptation, handling constant domain change using a single sample at a time presents two key challenges: efficiently stabilizing adaptation and effectively preventing catastrophic forgetting. This paper introduces a single-sample continual test-time adaptation (S-CoTTA) task to address these challenges. Existing works mainly either 1) apply continual test-time adaptation methods with an inefficient moving window that increases memory overhead or 2) filter out high-uncertainty samples to maintain stability. The former handled forgetting but failed to stabilize adaptation efficiently, while the latter neglected the catastrophic forgetting issues. We argue that both efficient tuning stabilization and forgetting prevention should be addressed simultaneously. To this end, we proposed a novel Efficient Buffer and Resetting (EBaR) method for S-CoTTA. EBaR employs a novel memory-efficient buffer to store samples based on their uncertainty levels and utilizes them to update the model with different losses to enhance stability. EBaR also incorporates a novel elastic resetting unit to dynamically reset the parameters based on their sensitivity to domain shift. The elastic resetting strategy effectively mitigates catastrophic forgetting while retaining useful target domain knowledge. Comprehensive experimental evaluations demonstrate the effectiveness and efficiency of both components. Combining their benefits, EBaR surpasses state-of-the-art methods across multiple datasets, including CIFAR10-C, CIFAR100-C, ImageNet-C, and CCC, for the S-CoTTA task.