
摘要
尽管在盲超分辨率领域已有诸多尝试,旨在恢复具有未知且复杂退化特征的低分辨率图像,但现有方法在处理真实世界中普遍存在的退化图像方面仍存在显著不足。本文将强大的ESRGAN模型拓展为一种实用的图像恢复应用(即Real-ESRGAN),该模型仅使用纯合成数据进行训练。具体而言,我们引入了一种高阶退化建模过程,以更准确地模拟真实世界中复杂的退化现象。同时,我们在合成过程中充分考虑了常见的振铃效应和过冲伪影问题。此外,我们采用带有谱归一化(spectral normalization)的U-Net判别器,以增强判别器的表达能力并稳定训练过程的动力学特性。大量实验对比表明,Real-ESRGAN在多个真实图像数据集上均展现出优于先前方法的视觉重建效果。此外,我们还提供了高效的实现方案,支持在训练过程中实时生成训练样本对。
代码仓库
sberbank-ai/real-esrgan
pytorch
GitHub 中提及
bayuudachi/Real-esr-gan
pytorch
GitHub 中提及
final-0/Real-ESRGAN
pytorch
GitHub 中提及
final-0/Real-ESRGAN-bicubic
pytorch
GitHub 中提及
xinntao/Real-ESRGAN
官方
pytorch
GitHub 中提及
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| video-super-resolution-on-msu-super-1 | Real-ESRGAN + uavs3e | BSQ-rate over ERQA: 7.225 BSQ-rate over LPIPS: 2.633 BSQ-rate over MS-SSIM: 4.612 BSQ-rate over PSNR: 15.144 BSQ-rate over Subjective Score: 1.417 BSQ-rate over VMAF: 2.122 |
| video-super-resolution-on-msu-super-1 | Real-ESRGAN + x264 | BSQ-rate over ERQA: 5.58 BSQ-rate over LPIPS: 0.733 BSQ-rate over MS-SSIM: 0.881 BSQ-rate over PSNR: 7.874 BSQ-rate over Subjective Score: 0.335 BSQ-rate over VMAF: 0.698 |
| video-super-resolution-on-msu-super-1 | Real-ESRGAN + x265 | BSQ-rate over ERQA: 6.328 BSQ-rate over LPIPS: 12.689 BSQ-rate over MS-SSIM: 5.393 BSQ-rate over PSNR: 8.113 BSQ-rate over Subjective Score: 0.64 BSQ-rate over VMAF: 1.464 |
| video-super-resolution-on-msu-super-1 | Real-ESRGAN + aomenc | BSQ-rate over ERQA: 11.584 BSQ-rate over LPIPS: 11.957 BSQ-rate over MS-SSIM: 6.857 BSQ-rate over PSNR: 15.144 BSQ-rate over Subjective Score: 1.398 BSQ-rate over VMAF: 2.712 |
| video-super-resolution-on-msu-super-1 | Real-ESRGAN + vvenc | BSQ-rate over ERQA: 6.712 BSQ-rate over LPIPS: 12.744 BSQ-rate over MS-SSIM: 5.95 BSQ-rate over PSNR: 14.561 BSQ-rate over VMAF: 3.8 |
| video-super-resolution-on-msu-video-upscalers | RealEsrgan-F | LPIPS: 0.185 PSNR: 28.82 SSIM: 0.850 |
| video-super-resolution-on-msu-video-upscalers | RealEsrgan | LPIPS: 0.181 PSNR: 29.14 SSIM: 0.855 |
| video-super-resolution-on-msu-video-upscalers | RealEsrgan-A | LPIPS: 0.244 PSNR: 28.71 SSIM: 0.830 |
| video-super-resolution-on-msu-video-upscalers | RealEsrgan-V | LPIPS: 0.333 PSNR: 25.52 SSIM: 0.795 |
| video-super-resolution-on-msu-video-upscalers | RealEsrnet | LPIPS: 0.296 PSNR: 30.52 SSIM: 0.878 |
| video-super-resolution-on-msu-video-upscalers | RealEsrnet-F | LPIPS: 0.280 PSNR: 30.01 SSIM: 0.868 |
| video-super-resolution-on-msu-vsr-benchmark | Real-ESRnet | 1 - LPIPS: 0.871 ERQAv1.0: 0.598 FPS: 1.019 PSNR: 27.195 QRCRv1.0: 0 SSIM: 0.824 Subjective score: 3.697 |
| video-super-resolution-on-msu-vsr-benchmark | Real-ESRGAN | 1 - LPIPS: 0.895 ERQAv1.0: 0.663 FPS: 1.01 PSNR: 24.441 QRCRv1.0: 0 SSIM: 0.774 Subjective score: 5.392 |