摘要
本文提出了一种基于新型循环网络架构的快速视频显著目标检测模型，命名为金字塔空洞双向卷积LSTM（Pyramid Dilated Bidirectional ConvLSTM，简称PDB-ConvLSTM）。首先，设计了一个金字塔空洞卷积（Pyramid Dilated Convolution, PDC）模块，用于同时提取多尺度的空间特征。随后，将这些空间特征进行拼接，并输入到扩展的深层双向卷积LSTM（Deeper Bidirectional ConvLSTM，DB-ConvLSTM）中，以学习时空信息。前向和后向的ConvLSTM单元被分别置于两层中，并以级联方式连接，从而促进双向流之间的信息交互，实现更深层次的特征提取。为进一步增强DB-ConvLSTM的表达能力，引入类似PDC的结构，通过多个空洞化的DB-ConvLSTM模块来捕捉多尺度的时空特征。大量实验结果表明，所提方法在性能上显著优于以往的视频显著性检测模型，在单张GPU上实现了高达20帧/秒的实时处理速度。以无监督视频对象分割为例，结合基于条件随机场（CRF）的后处理，该模型在两个主流基准数据集上均取得了当前最优的性能，充分验证了其优越的检测能力与广泛的应用潜力。
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| unsupervised-video-object-segmentation-on-10 | PDB | F: 74.5 G: 75.9 J: 77.2 |
| unsupervised-video-object-segmentation-on-11 | PDB | J: 74.0 |
| unsupervised-video-object-segmentation-on-12 | PDB | J: 65.5 |
| unsupervised-video-object-segmentation-on-4 | PDB | F-measure (Mean): 57.0 F-measure (Recall): 60.2 Ju0026F: 55.1 Jaccard (Mean): 53.2 Jaccard (Recall): 58.9 |
| unsupervised-video-object-segmentation-on-5 | PDB | F-measure (Decay): 3.7 F-measure (Mean): 43.0 F-measure (Recall): 44.6 Ju0026F: 40.4 Jaccard (Decay): 4.0 Jaccard (Mean): 37.7 Jaccard (Recall): 42.6 |
| video-salient-object-detection-on-davis-2016 | PDB | AVERAGE MAE: 0.028 MAX E-MEASURE: 0.951 S-Measure: 0.882 |
| video-salient-object-detection-on-davsod | PDB | Average MAE: 0.114 S-Measure: 0.706 max E-Measure: 0.749 max F-Measure: 0.591 |
| video-salient-object-detection-on-davsod-1 | PDB | Average MAE: 0.132 S-Measure: 0.649 max E-measure: 0.698 |
| video-salient-object-detection-on-davsod-2 | PDB | Average MAE: 0.107 S-Measure: 0.608 max E-measure: 0.678 |
| video-salient-object-detection-on-fbms-59 | PDB | AVERAGE MAE: 0.064 MAX F-MEASURE: 0.821 S-Measure: 0.851 |
| video-salient-object-detection-on-mcl | PDB | AVERAGE MAE: 0.021 MAX E-MEASURE: 0.911 S-Measure: 0.856 |
| video-salient-object-detection-on-segtrack-v2 | PDB | AVERAGE MAE: 0.024 S-Measure: 0.864 max E-measure: 0.935 |
| video-salient-object-detection-on-uvsd | PDB | Average MAE: 0.018 S-Measure: 0.901 max E-measure: 0.975 |
| video-salient-object-detection-on-visal | PDB | Average MAE: 0.032 S-Measure: 0.907 max E-measure: 0.846 |
| video-salient-object-detection-on-vos-t | PDB | Average MAE: 0.078 S-Measure: 0.818 max E-measure: 0.837 |