Enhanced Motion-aware Latent Diffusion Models for Video Frame Interpolation

摘要

The objective of video frame interpolation (VFI) methods is to enhance video fluency and visual quality by generating intermediate frames between consecutive original frames based on the source video. Recently, diffusion-based VFI methods have made promising progresses, with generated results performing well in perceptual quality. However, these methods have not fully explored how to effectively leverage external motion priors to enhance the model's ability to estimate motion information between adjacent frames, which is crucial for VFI models to avoid generating blurry results due to the motion ambiguity. In this paper, we propose an Enhanced Motion-Aware latent Diffusion model ( EMADiff ) for video frame interpolation. Specifically, we integrate motion priors into the decoder of vector-quantized enhanced motion-aware GAN to guide the information propagation during RGB interpolated frame reconstruction. Furthermore, we propose enhanced motion-aware noising and de-noising procedures. By reducing the discrepancy in attention to motion priors between the forward and reverse processes, our EMADiff effectively utilizes motion priors, alleviates motion ambiguity, and generates realistic content. Comprehensive experiments on benchmark datasets show EMADiff achieves state-of-the-art performance, surpassing existing approaches and producing visually plausible and content-clear results.