Heo Yuk ; Koh Yeong Jun ; Kim Chang-Su

Abstract

An interactive video object segmentation algorithm, which takes scribbleannotations on query objects as input, is proposed in this paper. We develop adeep neural network, which consists of the annotation network (A-Net) and thetransfer network (T-Net). First, given user scribbles on a frame, A-Net yieldsa segmentation result based on the encoder-decoder architecture. Second, T-Nettransfers the segmentation result bidirectionally to the other frames, byemploying the global and local transfer modules. The global transfer moduleconveys the segmentation information in an annotated frame to a target frame,while the local transfer module propagates the segmentation information in atemporally adjacent frame to the target frame. By applying A-Net and T-Netalternately, a user can obtain desired segmentation results with minimalefforts. We train the entire network in two stages, by emulating user scribblesand employing an auxiliary loss. Experimental results demonstrate that theproposed interactive video object segmentation algorithm outperforms thestate-of-the-art conventional algorithms. Codes and models are available athttps://github.com/yuk6heo/IVOS-ATNet.