Cheng Xuelian ; Xiong Huan ; Fan Deng-Ping ; Zhong Yiran ; Harandi Mehrtash ; Drummond Tom ; Ge Zongyuan

Abstract

We propose a new video camouflaged object detection (VCOD) framework that canexploit both short-term dynamics and long-term temporal consistency to detectcamouflaged objects from video frames. An essential property of camouflagedobjects is that they usually exhibit patterns similar to the background andthus make them hard to identify from still images. Therefore, effectivelyhandling temporal dynamics in videos becomes the key for the VCOD task as thecamouflaged objects will be noticeable when they move. However, current VCODmethods often leverage homography or optical flows to represent motions, wherethe detection error may accumulate from both the motion estimation error andthe segmentation error. On the other hand, our method unifies motion estimationand object segmentation within a single optimization framework. Specifically,we build a dense correlation volume to implicitly capture motions betweenneighbouring frames and utilize the final segmentation supervision to optimizethe implicit motion estimation and segmentation jointly. Furthermore, toenforce temporal consistency within a video sequence, we jointly utilize aspatio-temporal transformer to refine the short-term predictions. Extensiveexperiments on VCOD benchmarks demonstrate the architectural effectiveness ofour approach. We also provide a large-scale VCOD dataset named MoCA-Mask withpixel-level handcrafted ground-truth masks and construct a comprehensive VCODbenchmark with previous methods to facilitate research in this direction.Dataset Link: https://xueliancheng.github.io/SLT-Net-project.