
摘要
误差传播是在线半监督视频对象分割中一个普遍且关键的问题。为有效抑制误差传播,我们提出一种具有高可靠性的修正机制。核心思想是将修正过程与传统的掩码传播过程解耦,并基于可靠的线索进行独立处理。为此,我们引入两种调制模块:传播调制模块与修正调制模块,分别根据局部时序相关性以及可靠的参考信息,对目标帧的嵌入特征进行通道级重校准。具体而言,我们采用级联式的传播-修正架构来集成这两个调制模块,从而避免传播调制模块对可靠修正调制模块产生干扰或覆盖其效果。尽管带有真实标注的参考帧能够提供可靠的线索,但其与目标帧之间可能存在显著差异,从而引入不确定或不完整的时序关联。为此,我们通过向一个持续维护的特征块池中补充可靠的特征区域,增强参考线索的表达能力,使调制模块能够获得更加全面且富有表现力的对象表征。此外,我们设计了一种可靠性过滤机制,用于筛选出可靠的特征块,并将其传递至后续帧中。所提出的模型在YouTube-VOS 2018/2019以及DAVIS 17-Val/Test基准上均取得了当前最优的性能表现。大量实验结果表明,该修正机制通过充分挖掘可靠引导信息,显著提升了模型的整体性能。代码已开源,地址为:https://github.com/JerryX1110/RPCMVOS。
代码仓库
jerryx1110/rpcmvos
官方
pytorch
GitHub 中提及
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| semi-supervised-video-object-segmentation-on-1 | RPCMVOS | F-measure (Mean): 82.6 Ju0026F: 79.2 Jaccard (Mean): 75.8 |
| semi-supervised-video-object-segmentation-on-1 | RPCMVOS-Full-Res | F-measure (Mean): 84.3 Ju0026F: 81 Jaccard (Mean): 77.6 |
| semi-supervised-video-object-segmentation-on-18 | RPCMVOS | F-Measure (Seen): 86.9 F-Measure (Unseen): 87.1 Jaccard (Seen): 82.6 Jaccard (Unseen): 79.1 Overall: 83.9 |
| video-object-segmentation-on-davis-2017-test-1 | RPCMVOS | F-measure: 82.6 Jaccard: 75.8 Mean Jaccard u0026 F-Measure: 79.2 |
| video-object-segmentation-on-youtube-vos | RPCMVOS | F-Measure (Seen): 87.7 F-Measure (Unseen): 86.7 Jaccard (Seen): 83.1 Overall: 84 Speed (FPS): 78.5 |
| video-object-segmentation-on-youtube-vos | RPCMVOS-MS | F-Measure (Seen): 87.9 F-Measure (Unseen): 86.9 Jaccard (Seen): 83.3 Jaccard (Unseen): 78.9 Overall: 84.3 |
| video-object-segmentation-on-youtube-vos-2019-2 | RPCMVOS | F-Measure (Seen): 86.9 F-Measure (Unseen): 87.1 Jaccard (Seen): 82.6 Jaccard (Unseen): 79.1 Mean Jaccard u0026 F-Measure: 83.9 |
| visual-object-tracking-on-davis-2016 | RPCMVOS | F-measure (Mean): 94 Ju0026F: 90.6 Jaccard (Mean): 87.1 |
| visual-object-tracking-on-davis-2017 | RPCMVOS | F-measure (Mean): 86 Ju0026F: 83.7 Jaccard (Mean): 81.3 |