3 个月前

ManTra-Net:用于检测与定位具有异常特征的图像伪造的操纵追踪网络

ManTra-Net:用于检测与定位具有异常特征的图像伪造的操纵追踪网络

摘要

为应对现实生活中常见的、涉及多种类型且常以复合方式出现的图像伪造问题,本文提出了一种统一的深度神经网络架构——ManTra-Net。与现有大多数方法不同,ManTra-Net是一种端到端网络,无需额外的预处理或后处理即可同时实现伪造检测与定位。该网络基于全卷积结构,能够处理任意尺寸的图像,并有效识别多种已知伪造类型,包括图像拼接(splicing)、复制-粘贴(copy-move)、内容删除(removal)、图像增强(enhancement)等,甚至可应对未知类型的伪造。本文的主要贡献有三点:首先,我们设计了一种简单而高效的自监督学习任务,通过分类385种不同的图像操作类型,学习鲁棒的图像篡改痕迹特征;其次,我们将伪造定位问题建模为局部异常检测任务,提出一种Z-score特征以捕捉局部异常,并设计了一种新型的长短期记忆(LSTM)机制来评估局部异常程度;最后,我们通过精心设计的消融实验,系统性地优化了所提出的网络结构。大量实验结果表明,ManTra-Net在单一伪造类型以及复杂组合伪造场景下均展现出优异的泛化能力、鲁棒性与性能优势,显著优于现有方法。

基准测试

基准方法指标
image-manipulation-detection-on-casia-osnMantra-Net
AUC: 0.763
F-score: 0.102
Intersection over Union: 0 .065
image-manipulation-detection-on-casia-osn-1ManTra-Net
AUC: 0.724
Intersection over Union: 0.080
f-Score: 0.048
image-manipulation-detection-on-casia-osn-2ManTra-Net
AUC: 0.763
Intersection over Union: 0.063
f-Score: 0.099
image-manipulation-detection-on-casia-osn-3ManTra-Net
AUC: 0.754
Intersection over Union: 0.063
f-Score: 0.099
image-manipulation-detection-on-casia-v1ManTraNet
AUC: .644
Balanced Accuracy: .500
image-manipulation-detection-on-cocoglideManTraNet
AUC: .778
Balanced Accuracy: .500
image-manipulation-detection-on-columbiaManTraNet
AUC: .810
Balanced Accuracy: .500
image-manipulation-detection-on-columbia-osnManTra-Net
AUC: 0.626
Intersection over Union: 0.056
f-Score: 0.103
image-manipulation-detection-on-columbia-osn-1ManTra-Net
AUC: 0.613
Intersection over Union: 0.125
f-Score: 0.199
image-manipulation-detection-on-columbia-osn-2ManTra-Net
AUC: 0.630
Intersection over Union: 0.052
f-Score: 0.098
image-manipulation-detection-on-columbia-osn-3ManTra-Net
AUC: 0.620
Intersection over Union: 0.056
f-Score: 0.103
image-manipulation-detection-on-coverageManTraNet
AUC: .760
Balanced Accuracy: .500
image-manipulation-detection-on-dso-1ManTraNet
AUC: .874
Balanced Accuracy: .500
image-manipulation-detection-on-dso-osnManTra-Net
AUC: 0.638
Intersection over Union: 0.071
f-Score: 0.109
image-manipulation-detection-on-dso-osn-1ManTra-Net
AUC: 0.582
Intersection over Union: 0.045
f-Score: 0.076
image-manipulation-detection-on-dso-osn-2ManTra-Net
AUC: 0.616
Intersection over Union: 0.052
f-Score: 0.081
image-manipulation-detection-on-dso-osn-3ManTra-Net
AUC: 0.606
Intersection over Union: 0.036
f-Score: 0.057
image-manipulation-detection-on-nist-osnManTra-Net
AUC: 0.652
Intersection over Union: 0.057
f-Score: 0.095
image-manipulation-detection-on-nist-osn-1ManTra-Net
AUC: 0.654
Intersection over Union: 0.057
f-Score: 0.095
image-manipulation-detection-on-nist-osn-2ManTra-Net
AUC: 0.702
Intersection over Union: 0.062
f-Score: 0.101
image-manipulation-detection-on-nist-osn-3ManTra-Net
AUC: 0.671
Intersection over Union: 0.053
f-Score: 0.088
image-manipulation-localization-on-casia-v1ManTraNet
Average Pixel F1(Fixed threshold): .180
image-manipulation-localization-on-cocoglideManTraNet
Average Pixel F1(Fixed threshold): .516
image-manipulation-localization-on-columbiaManTraNet
Average Pixel F1(Fixed threshold): .508
image-manipulation-localization-on-coverageManTraNet
Average Pixel F1(Fixed threshold): .317
image-manipulation-localization-on-dso-1ManTraNet
Average Pixel F1(Fixed threshold): .412

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程
即用型 GPU
最优价格
立即开始

Hyper Newsletters

订阅我们的最新资讯
我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新
邮件发送服务由 MailChimp 提供
ManTra-Net:用于检测与定位具有异常特征的图像伪造的操纵追踪网络 | 论文 | HyperAI超神经