ColorSurge: Bringing Vibrancy and Efficiency to Automatic Video Colorization via Dual-Branch Fusion

摘要

Automatic video colorization poses challenges, requiring efficient generation of results that ensure frame and color consistency. Previous video colorization works often suffer from issues such as color flickering, bleeding, artifacts, and low color richness due to the inherent ambiguity and limitations of the models. While diffusion-based video-to-video approaches can produce customized colorization models through fine-tuning, their high inference costs limit their suitability for real-time scenarios. To address these challenges, we propose ColorSurge, a lightweight network for efficient end-to-end video colorization. ColorSurge employs a dual-branch structure, consisting of a grayscale branch and a color branch. In the grayscale branch, we extract the semantic content of grayscale videos and reconstruct and output features at different spatial scales. In the color branch, we introduce learnable color tokens and fuse these multi-scale semantic features through stacked Color Alchemy Blocks (CABs). Within each CAB, we incorporate Color Spatial Transformer Blocks (CSTB) and Color Temporal Transformer Blocks (CTTB) to constrain the spatial harmony and temporal consistency of colors. Finally, we use a Color Mapper to unify the grayscale and color features, mapping them to obtain the final colorized video result. Extensive experiments demonstrate that our method significantly outperforms previous state-of-the-art models in both qualitative and quantitative evaluations. Our code and model are available at https://github.com/ABTols/ColorSurge.