FlowTrack: Integrating Adjacent-Frame Motion Tracking and Adaptive Prediction for Robust Semi-Supervised VOS

摘要

Semi-supervised Video Object Segmentation (VOS) aims to segment a user-specified object across all frames of a video using only the first frame's annotated mask. A key challenge in VOS is simultaneously preserving object identity and maintaining precise segmentation in dynamic scenes, especially during rapid motion. Many existing methods use previous frame masks as positional constraints, hindering segmentation of newly exposed regions-an issue known as over-suppression. To address these challenges, we propose FlowTrack, integrating two main components: an Adjacent-frame Motion Tracker (AMT) and an Adaptive Motion Predictor (AMP). AMT explicitly captures motion and positional information between adjacent frames and fuses it with historical target cues, improving segmentation constraint robustness and helping maintain stable object identity during rapid motion or significant deformation. However, relying primarily on historical masks and frame-level constraints may fail to accurately predict sudden changes in motion states, which still leads to over-suppression. To overcome this limitation, AMP predicts future states from historical motion data. Employing a learned state predictor and a Kalman-inspired recursive measurement fusion, AMP adapts to complex and abrupt motion changes. This dynamic prediction-update scheme refines segmentation boundaries, compensates for historical constraints, and effectively mitigates over-suppression. Experimental results on standard VOS benchmarks validate the effectiveness of the proposed FlowTrack framework in handling challenging dynamic scenes involving rapid motion and addressing the over-suppression issue.