3 个月前

Real-ESRGAN:仅使用纯合成数据训练真实世界盲超分辨率

Real-ESRGAN:仅使用纯合成数据训练真实世界盲超分辨率

摘要

尽管在盲超分辨率领域已有诸多尝试,旨在恢复具有未知且复杂退化特征的低分辨率图像,但现有方法在处理真实世界中普遍存在的退化图像方面仍存在显著不足。本文将强大的ESRGAN模型拓展为一种实用的图像恢复应用(即Real-ESRGAN),该模型仅使用纯合成数据进行训练。具体而言,我们引入了一种高阶退化建模过程,以更准确地模拟真实世界中复杂的退化现象。同时,我们在合成过程中充分考虑了常见的振铃效应和过冲伪影问题。此外,我们采用带有谱归一化(spectral normalization)的U-Net判别器,以增强判别器的表达能力并稳定训练过程的动力学特性。大量实验对比表明,Real-ESRGAN在多个真实图像数据集上均展现出优于先前方法的视觉重建效果。此外,我们还提供了高效的实现方案,支持在训练过程中实时生成训练样本对。

代码仓库

基准测试

基准方法指标
video-super-resolution-on-msu-super-1Real-ESRGAN + uavs3e
BSQ-rate over ERQA: 7.225
BSQ-rate over LPIPS: 2.633
BSQ-rate over MS-SSIM: 4.612
BSQ-rate over PSNR: 15.144
BSQ-rate over Subjective Score: 1.417
BSQ-rate over VMAF: 2.122
video-super-resolution-on-msu-super-1Real-ESRGAN + x264
BSQ-rate over ERQA: 5.58
BSQ-rate over LPIPS: 0.733
BSQ-rate over MS-SSIM: 0.881
BSQ-rate over PSNR: 7.874
BSQ-rate over Subjective Score: 0.335
BSQ-rate over VMAF: 0.698
video-super-resolution-on-msu-super-1Real-ESRGAN + x265
BSQ-rate over ERQA: 6.328
BSQ-rate over LPIPS: 12.689
BSQ-rate over MS-SSIM: 5.393
BSQ-rate over PSNR: 8.113
BSQ-rate over Subjective Score: 0.64
BSQ-rate over VMAF: 1.464
video-super-resolution-on-msu-super-1Real-ESRGAN + aomenc
BSQ-rate over ERQA: 11.584
BSQ-rate over LPIPS: 11.957
BSQ-rate over MS-SSIM: 6.857
BSQ-rate over PSNR: 15.144
BSQ-rate over Subjective Score: 1.398
BSQ-rate over VMAF: 2.712
video-super-resolution-on-msu-super-1Real-ESRGAN + vvenc
BSQ-rate over ERQA: 6.712
BSQ-rate over LPIPS: 12.744
BSQ-rate over MS-SSIM: 5.95
BSQ-rate over PSNR: 14.561
BSQ-rate over VMAF: 3.8
video-super-resolution-on-msu-video-upscalersRealEsrgan-F
LPIPS: 0.185
PSNR: 28.82
SSIM: 0.850
video-super-resolution-on-msu-video-upscalersRealEsrgan
LPIPS: 0.181
PSNR: 29.14
SSIM: 0.855
video-super-resolution-on-msu-video-upscalersRealEsrgan-A
LPIPS: 0.244
PSNR: 28.71
SSIM: 0.830
video-super-resolution-on-msu-video-upscalersRealEsrgan-V
LPIPS: 0.333
PSNR: 25.52
SSIM: 0.795
video-super-resolution-on-msu-video-upscalersRealEsrnet
LPIPS: 0.296
PSNR: 30.52
SSIM: 0.878
video-super-resolution-on-msu-video-upscalersRealEsrnet-F
LPIPS: 0.280
PSNR: 30.01
SSIM: 0.868
video-super-resolution-on-msu-vsr-benchmarkReal-ESRnet
1 - LPIPS: 0.871
ERQAv1.0: 0.598
FPS: 1.019
PSNR: 27.195
QRCRv1.0: 0
SSIM: 0.824
Subjective score: 3.697
video-super-resolution-on-msu-vsr-benchmarkReal-ESRGAN
1 - LPIPS: 0.895
ERQAv1.0: 0.663
FPS: 1.01
PSNR: 24.441
QRCRv1.0: 0
SSIM: 0.774
Subjective score: 5.392

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程
即用型 GPU
最优价格
立即开始

Hyper Newsletters

订阅我们的最新资讯
我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新
邮件发送服务由 MailChimp 提供