
摘要
为应对现实生活中常见的、涉及多种类型且常以复合方式出现的图像伪造问题,本文提出了一种统一的深度神经网络架构——ManTra-Net。与现有大多数方法不同,ManTra-Net是一种端到端网络,无需额外的预处理或后处理即可同时实现伪造检测与定位。该网络基于全卷积结构,能够处理任意尺寸的图像,并有效识别多种已知伪造类型,包括图像拼接(splicing)、复制-粘贴(copy-move)、内容删除(removal)、图像增强(enhancement)等,甚至可应对未知类型的伪造。本文的主要贡献有三点:首先,我们设计了一种简单而高效的自监督学习任务,通过分类385种不同的图像操作类型,学习鲁棒的图像篡改痕迹特征;其次,我们将伪造定位问题建模为局部异常检测任务,提出一种Z-score特征以捕捉局部异常,并设计了一种新型的长短期记忆(LSTM)机制来评估局部异常程度;最后,我们通过精心设计的消融实验,系统性地优化了所提出的网络结构。大量实验结果表明,ManTra-Net在单一伪造类型以及复杂组合伪造场景下均展现出优异的泛化能力、鲁棒性与性能优势,显著优于现有方法。
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| image-manipulation-detection-on-casia-osn | Mantra-Net | AUC: 0.763 F-score: 0.102 Intersection over Union: 0 .065 |
| image-manipulation-detection-on-casia-osn-1 | ManTra-Net | AUC: 0.724 Intersection over Union: 0.080 f-Score: 0.048 |
| image-manipulation-detection-on-casia-osn-2 | ManTra-Net | AUC: 0.763 Intersection over Union: 0.063 f-Score: 0.099 |
| image-manipulation-detection-on-casia-osn-3 | ManTra-Net | AUC: 0.754 Intersection over Union: 0.063 f-Score: 0.099 |
| image-manipulation-detection-on-casia-v1 | ManTraNet | AUC: .644 Balanced Accuracy: .500 |
| image-manipulation-detection-on-cocoglide | ManTraNet | AUC: .778 Balanced Accuracy: .500 |
| image-manipulation-detection-on-columbia | ManTraNet | AUC: .810 Balanced Accuracy: .500 |
| image-manipulation-detection-on-columbia-osn | ManTra-Net | AUC: 0.626 Intersection over Union: 0.056 f-Score: 0.103 |
| image-manipulation-detection-on-columbia-osn-1 | ManTra-Net | AUC: 0.613 Intersection over Union: 0.125 f-Score: 0.199 |
| image-manipulation-detection-on-columbia-osn-2 | ManTra-Net | AUC: 0.630 Intersection over Union: 0.052 f-Score: 0.098 |
| image-manipulation-detection-on-columbia-osn-3 | ManTra-Net | AUC: 0.620 Intersection over Union: 0.056 f-Score: 0.103 |
| image-manipulation-detection-on-coverage | ManTraNet | AUC: .760 Balanced Accuracy: .500 |
| image-manipulation-detection-on-dso-1 | ManTraNet | AUC: .874 Balanced Accuracy: .500 |
| image-manipulation-detection-on-dso-osn | ManTra-Net | AUC: 0.638 Intersection over Union: 0.071 f-Score: 0.109 |
| image-manipulation-detection-on-dso-osn-1 | ManTra-Net | AUC: 0.582 Intersection over Union: 0.045 f-Score: 0.076 |
| image-manipulation-detection-on-dso-osn-2 | ManTra-Net | AUC: 0.616 Intersection over Union: 0.052 f-Score: 0.081 |
| image-manipulation-detection-on-dso-osn-3 | ManTra-Net | AUC: 0.606 Intersection over Union: 0.036 f-Score: 0.057 |
| image-manipulation-detection-on-nist-osn | ManTra-Net | AUC: 0.652 Intersection over Union: 0.057 f-Score: 0.095 |
| image-manipulation-detection-on-nist-osn-1 | ManTra-Net | AUC: 0.654 Intersection over Union: 0.057 f-Score: 0.095 |
| image-manipulation-detection-on-nist-osn-2 | ManTra-Net | AUC: 0.702 Intersection over Union: 0.062 f-Score: 0.101 |
| image-manipulation-detection-on-nist-osn-3 | ManTra-Net | AUC: 0.671 Intersection over Union: 0.053 f-Score: 0.088 |
| image-manipulation-localization-on-casia-v1 | ManTraNet | Average Pixel F1(Fixed threshold): .180 |
| image-manipulation-localization-on-cocoglide | ManTraNet | Average Pixel F1(Fixed threshold): .516 |
| image-manipulation-localization-on-columbia | ManTraNet | Average Pixel F1(Fixed threshold): .508 |
| image-manipulation-localization-on-coverage | ManTraNet | Average Pixel F1(Fixed threshold): .317 |
| image-manipulation-localization-on-dso-1 | ManTraNet | Average Pixel F1(Fixed threshold): .412 |