摘要
近期最先进的超分辨率方法在理想数据集上取得了令人瞩目的性能表现,无论在模糊或噪声条件下均表现出色。然而,这些方法在真实世界图像超分辨率任务中往往表现不佳,主要原因在于其训练数据通常通过从高质量图像中采用简单的双三次下采样方式构建低分辨率(LR)与高分辨率(HR)图像对,这种处理方式容易丢失与频率相关的细节信息。为解决这一问题,本文提出一种面向真实世界图像的新型退化建模框架,通过估计多种模糊核以及真实噪声分布,更准确地模拟真实图像退化过程。基于该退化框架,我们能够生成与真实世界图像具有相同分布特性的低分辨率图像。在此基础上,我们进一步提出一种面向真实世界超分辨率的新型模型,旨在提升视觉感知质量。在合成噪声数据和真实世界图像上的大量实验结果表明,所提方法显著优于现有最先进方法,在降低噪声水平和提升视觉质量方面均表现优异。此外,该方法在NTIRE 2020真实世界超分辨率挑战赛的两个赛道中均获得冠军,性能远超其他参赛方法,展现出显著优势。
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| video-super-resolution-on-msu-super-1 | RealSR + uavs3e | BSQ-rate over ERQA: 1.943 BSQ-rate over LPIPS: 1.149 BSQ-rate over MS-SSIM: 1.441 BSQ-rate over PSNR: 14.741 BSQ-rate over Subjective Score: 0.639 BSQ-rate over VMAF: 2.253 |
| video-super-resolution-on-msu-super-1 | RealSR + x265 | BSQ-rate over ERQA: 1.622 BSQ-rate over LPIPS: 1.206 BSQ-rate over MS-SSIM: 1.033 BSQ-rate over PSNR: 1.064 BSQ-rate over Subjective Score: 0.502 BSQ-rate over VMAF: 1.617 |
| video-super-resolution-on-msu-super-1 | RealSR + vvenc | BSQ-rate over ERQA: 21.965 BSQ-rate over LPIPS: 18.344 BSQ-rate over MS-SSIM: 11.643 BSQ-rate over PSNR: 15.144 BSQ-rate over VMAF: 10.67 |
| video-super-resolution-on-msu-super-1 | RealSR + x264 | BSQ-rate over ERQA: 0.77 BSQ-rate over LPIPS: 0.591 BSQ-rate over MS-SSIM: 0.487 BSQ-rate over PSNR: 0.675 BSQ-rate over Subjective Score: 0.196 BSQ-rate over VMAF: 0.775 |
| video-super-resolution-on-msu-super-1 | RealSR + aomenc | BSQ-rate over ERQA: 6.762 BSQ-rate over LPIPS: 10.915 BSQ-rate over MS-SSIM: 5.463 BSQ-rate over PSNR: 15.144 BSQ-rate over Subjective Score: 0.843 BSQ-rate over VMAF: 4.283 |
| video-super-resolution-on-msu-video-upscalers | RealSR | LPIPS: 0.220 PSNR: 30.64 SSIM: 0.900 |
| video-super-resolution-on-msu-vsr-benchmark | RealSR | 1 - LPIPS: 0.911 ERQAv1.0: 0.69 FPS: 0.352 PSNR: 25.989 QRCRv1.0: 0 SSIM: 0.767 Subjective score: 5.286 |