Command Palette
Search for a command to run...
Qiang Wang; Li Zhang; Luca Bertinetto; Weiming Hu; Philip H.S. Torr
Abstract
In this paper we illustrate how to perform both visual object tracking and semi-supervised video object segmentation, in real-time, with a single simple approach. Our method, dubbed SiamMask, improves the offline training procedure of popular fully-convolutional Siamese approaches for object tracking by augmenting their loss with a binary segmentation task. Once trained, SiamMask solely relies on a single bounding box initialisation and operates online, producing class-agnostic object segmentation masks and rotated bounding boxes at 55 frames per second. Despite its simplicity, versatility and fast speed, our strategy allows us to establish a new state of the art among real-time trackers on VOT-2018, while at the same time demonstrating competitive performance and the best speed for the semi-supervised video object segmentation task on DAVIS-2016 and DAVIS-2017. The project website is http://www.robots.ox.ac.uk/~qwang/SiamMask.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| semi-supervised-video-object-segmentation-on-1 | SiamMask | F-measure (Decay): 22.4 F-measure (Mean): 45.8 F-measure (Recall): 45.3 Ju0026F: 43.2 Jaccard (Decay): 21.9 Jaccard (Mean): 40.6 Jaccard (Recall): 44.5 |
| video-object-tracking-on-nv-vot211 | SiamMask | AUC: 35.14 Precision: 46.49 |
| visual-object-tracking-on-davis-2016 | SiamMask | F-measure (Decay): 2.1 F-measure (Mean): 67.8 F-measure (Recall): 79.8 Ju0026F: 69.75 Jaccard (Decay): 3.0 Jaccard (Mean): 71.7 Jaccard (Recall): 86.8 |
| visual-object-tracking-on-davis-2017 | SiamMask | F-measure (Decay): 20.9 F-measure (Mean): 58.5 F-measure (Recall): 67.5 Ju0026F: 56.4 Jaccard (Decay): 19.3 Jaccard (Mean): 54.3 Jaccard (Recall): 62.8 |
| visual-object-tracking-on-vot201718 | SiamMask | Expected Average Overlap (EAO): 0.380 |
| visual-object-tracking-on-youtube-vos | SiamMask | F-Measure (Seen): 58.2 F-Measure (Unseen): 47.7 Jaccard (Seen): 54.3 Jaccard (Unseen): 45.1 O (Average of Measures): 52.8 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.