VRMusicStage: A System for Converting Fixed-Camera Music Stage Videos into Immersive VR Content

摘要

Recent advances in VR have created a growing demand for immersive content, especially in viewing music performances. However, most existing videos are captured with a narrow field of view, limiting their applicability in VR environments. In this paper, we present a practical system that converts fixed-camera music performance videos into immersive VR experiences via high-resolution video outpainting. Our method leverages pre-trained text-to-image diffusion models with multi-conditioning based on ControlNet to generate spatially and temporally consistent frames at high resolution. While our approach builds on existing components, we introduce a novel orchestration of these tools tailored specifically for immersive video generation, requiring no additional training and running efficiently on consumer GPUs. The system supports videos of arbitrary length via frame-by-frame processing and produces seamless 8K outputs through lightweight post-processing. Extensive experiments and user studies demonstrate that our system outperforms state-of-the-art methods in perceptual quality and viewer immersion, offering a scalable pathway for repurposing conventional footage into high-fidelity VR content.