摘要
我们提出了一种新的双边参考框架（BiRefNet），用于高分辨率二值图像分割（DIS）。该框架包含两个核心组件：定位模块（LM）和重建模块（RM），其中我们提出了双边参考（BiRef）。定位模块利用全局语义信息辅助目标定位。在重建模块中，我们利用双边参考进行重建过程，其中图像的层次化块提供源参考，而梯度图则作为目标参考。这些组件协同工作，生成最终的预测图。此外，我们引入了辅助梯度监督机制，以增强对细节更精细区域的关注。为进一步提高地图质量和训练过程，我们还详细介绍了针对二值图像分割的实际训练策略。为了验证我们的方法具有广泛的适用性，我们在四个任务上进行了大量实验，结果表明BiRefNet表现出显著的性能，在所有基准测试中均优于特定任务的最先进方法。我们的代码已发布在 https://github.com/ZhengPeng7/BiRefNet。
代码仓库
zhengpeng7/birefnet
官方
pytorch
GitHub 中提及
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| camouflaged-object-segmentation-on-camo | BiRefNet | MAE: 0.030 S-Measure: 0.904 Weighted F-Measure: 0.890 |
| camouflaged-object-segmentation-on-chameleon | BiRefNet | MAE: 0.015 S-measure: 0.932 weighted F-measure: 0.914 |
| camouflaged-object-segmentation-on-cod | BiRefNet | MAE: 0.014 S-Measure: 0.913 Weighted F-Measure: 0.874 |
| camouflaged-object-segmentation-on-nc4k | BiRefNet | MAE: 0.023 S-measure: 0.914 weighted F-measure: 0.894 |
| dichotomous-image-segmentation-on-dis-te1 | BiRefNet | E-measure: 0.908 HCE: 106 MAE: 0.038 S-Measure: 0.882 max F-Measure: 0.855 weighted F-measure: 0.814 |
| dichotomous-image-segmentation-on-dis-te2 | BiRefNet | E-measure: 0.935 HCE: 265 MAE: 0.035 S-Measure: 0.904 max F-Measure: 0.898 weighted F-measure: 0.863 |
| dichotomous-image-segmentation-on-dis-te3 | BiRefNet | E-measure: 0.952 HCE: 573 MAE: 0.030 S-Measure: 0.918 max F-Measure: 0.923 weighted F-measure: 0.891 |
| dichotomous-image-segmentation-on-dis-te4 | BiRefNet | E-measure: 0.937 HCE: 2746 MAE: 0.040 S-Measure: 0.898 max F-Measure: 0.900 weighted F-measure: 0.861 |
| dichotomous-image-segmentation-on-dis-vd | BiRefNet | E-measure: 0.928 HCE: 1006 MAE: 0.038 S-Measure: 0.898 max F-Measure: 0.889 weighted F-measure: 0.853 |
| rgb-salient-object-detection-on-davis-s | BiRefNet (DUTS, HRSOD) | F-measure: 0.976 MAE: 0.006 S-measure: 0.973 |
| rgb-salient-object-detection-on-davis-s | BiRefNet (DUTS, HRSOD, UHRSD) | F-measure: 0.979 MAE: 0.006 S-measure: 0.975 |
| rgb-salient-object-detection-on-davis-s | BiRefNet (DUTS) | F-measure: 0.966 MAE: 0.008 S-measure: 0.967 |
| rgb-salient-object-detection-on-davis-s | BiRefNet (HRSOD, UHRSD) | F-measure: 0.980 MAE: 0.006 S-measure: 0.976 |
| rgb-salient-object-detection-on-davis-s | BiRefNet (DUTS, UHRSD) | F-measure: 0.977 MAE: 0.006 S-measure: 0.975 |
| rgb-salient-object-detection-on-hrsod | BiRefNet (DUTS, UHRSD) | MAE: 0.014 S-Measure: 0.959 max F-Measure: 0.958 |
| rgb-salient-object-detection-on-hrsod | BiRefNet (DUTS, HRSOD) | MAE: 0.011 S-Measure: 0.962 max F-Measure: 0.963 |
| rgb-salient-object-detection-on-hrsod | BiRefNet (DUTS) | MAE: 0.014 S-Measure: 0.957 max F-Measure: 0.958 |
| rgb-salient-object-detection-on-hrsod | BiRefNet (HRSOD, UHRSD) | MAE: 0.016 S-Measure: 0.956 max F-Measure: 0.953 |
| rgb-salient-object-detection-on-hrsod | BiRefNet (DUTS, HRSOD, UHRSD) | MAE: 0.013 S-Measure: 0.962 max F-Measure: 0.961 |
| rgb-salient-object-detection-on-uhrsd | BiRefNet (HRSOD, UHRSD) | MAE: 0.019 S-Measure: 0.952 max F-Measure: 0.958 |
| rgb-salient-object-detection-on-uhrsd | BiRefNet (DUTS, UHRSD) | MAE: 0.019 S-Measure: 0.952 max F-Measure: 0.960 |
| rgb-salient-object-detection-on-uhrsd | BiRefNet (DUTS, HRSOD) | MAE: 0.024 S-Measure: 0.937 max F-Measure: 0.942 |
| rgb-salient-object-detection-on-uhrsd | BiRefNet (DUTS, HRSOD, UHRSD) | MAE: 0.016 S-Measure: 0.957 max F-Measure: 0.963 |
| rgb-salient-object-detection-on-uhrsd | BiRefNet (DUTS) | MAE: 0.030 S-Measure: 0.931 max F-Measure: 0.933 |
| salient-object-detection-on-dut-omron | BiRefNet (HRSOD, UHRSD) | F-measure: 0.810 MAE: 0.040 S-Measure: 0.864 Weighted F-Measure: 0.790 mean E-Measure: 0.879 mean F-Measure: 0.801 |
| salient-object-detection-on-dut-omron | BiRefNet (DUTS, UHRSD) | F-measure: 0.837 MAE: 0.036 S-Measure: 0.881 Weighted F-Measure: 0.815 mean E-Measure: 0.896 mean F-Measure: 0.825 |
| salient-object-detection-on-dut-omron | BiRefNet (DUTS, HRSOD) | F-measure: 0.818 MAE: 0.040 S-Measure: 0.868 Weighted F-Measure: 0.800 mean E-Measure: 0.882 mean F-Measure: 0.809 |
| salient-object-detection-on-dut-omron | BiRefNet (DUTS, HRSOD, UHRSD) | F-measure: 0.839 MAE: 0.038 S-Measure: 0.882 Weighted F-Measure: 0.815 mean E-Measure: 0.896 mean F-Measure: 0.825 |
| salient-object-detection-on-dut-omron | BiRefNet (DUTS) | F-measure: 0.813 MAE: 0.040 S-Measure: 0.868 Weighted F-Measure: 0.792 mean E-Measure: 0.878 mean F-Measure: 0.802 |
| salient-object-detection-on-duts-te | BiRefNet (DUTS, UHRSD) | MAE: 0.018 S-Measure: 0.942 Weighted F-Measure: 0.919 max F-measure: 0.942 mean E-Measure: 0.961 mean F-Measure: 0.925 |
| salient-object-detection-on-duts-te | BiRefNet (DUTS) | MAE: 0.019 S-Measure: 0.939 Weighted F-Measure: 0.913 max F-measure: 0.937 mean E-Measure: 0.958 mean F-Measure: 0.919 |
| salient-object-detection-on-duts-te | BiRefNet (DUTS, HRSOD, UHRSD) | MAE: 0.018 S-Measure: 0.944 Weighted F-Measure: 0.920 max F-measure: 0.943 mean E-Measure: 0.962 mean F-Measure: 0.925 |
| salient-object-detection-on-duts-te | BiRefNet (DUTS, HRSOD) | MAE: 0.018 S-Measure: 0.938 Weighted F-Measure: 0.918 max F-measure: 0.935 mean E-Measure: 0.960 mean F-Measure: 0.923 |
| salient-object-detection-on-duts-te | BiRefNet (HRSOD, UHRSD) | MAE: 0.020 S-Measure: 0.933 Weighted F-Measure: 0.907 max F-measure: 0.928 mean E-Measure: 0.954 mean F-Measure: 0.913 |