
摘要
近期研究通过卷积神经网络(CNN)显著提升了单图像超分辨率(Single-Image Super-Resolution, SR)的性能。尽管对于给定的输入图像可能存在多种高分辨率(HR)重建结果,但大多数现有的基于CNN的SR方法在推理阶段并未探索这些替代解。传统上,获取多样化SR结果的方法是训练多个具有不同损失权重的SR模型,并通过模型组合来实现。然而,本文提出一种更高效的方法:通过多任务学习机制,仅需训练一个可调节的单一SR模型,即可在多种损失组合下进行优化。具体而言,在训练过程中,我们采用条件化目标函数,该目标函数为不同特征层级上多个感知损失的加权和,其中权重根据预设条件动态变化,这一权重集合被定义为“风格控制器”(style controller)。此外,我们设计了一种适配该训练范式的网络架构——基于空间特征变换层的残差嵌套密集块(Residual-in-Residual Dense Block)。在推理阶段,所训练的模型能够根据风格控制图生成局部差异化的输出结果。大量实验表明,所提出的SR模型能够在不引入伪影的前提下生成多种理想的重建结果,并在定量指标上达到与当前最先进方法相当的性能水平。
代码仓库
seungho-snu/fxsr
官方
pytorch
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| image-super-resolution-on-bsd100-4x-upscaling | FxSR-PD t=0.8 | LPIPS: 0.1572 PSNR: 24.77 SSIM: 0.6817 |
| image-super-resolution-on-bsd100-4x-upscaling | FxSR-PD t=0.0 | LPIPS: 0.3433 PSNR: 26.38 SSIM: 0.738 |
| image-super-resolution-on-bsd100-8x-upscaling | FxSR-PD t=0.0 | DISTS: 0.2753 LPIPS: 0.5079 LRPSNR: 47.12 NIQE: 5.49 PSNR: 23.6 SSIM: 0.5728 |
| image-super-resolution-on-bsd100-8x-upscaling | FxSR-PD t=0.8 | DISTS: 0.1972 LPIPS: 0.3129 LRPSNR: 42.41 NIQE: 4.58 PSNR: 21.93 SSIM: 0.5039 |
| image-super-resolution-on-div2k-val-4x | FxSR-PD t=0.8 | DISTS: 0.0513 LPIPS: 0.1028 LRPSNR: 50.54 NIQE: 2.81 PSNR: 27.51 SSIM: 0.789 |
| image-super-resolution-on-div2k-val-4x | FxSR-PD t=0.0 | DISTS: 0.1169 LPIPS: 0.239 LRPSNR: 53.3 NIQE: 4.11 PSNR: 29.24 SSIM: 0.8383 |
| image-super-resolution-on-div2k-val-8x | FxSR-PD t=0.8 | DISTS: 0.119 LPIPS: 0.2403 LRPSNR: 42.66 NIQE: 3.61 PSNR: 23.56 SSIM: 0.6241 |
| image-super-resolution-on-div2k-val-8x | FxSR-PD t=0.0 | DISTS: 0.1953 LPIPS: 0.3857 LRPSNR: 46.96 NIQE: 4.41 PSNR: 25.6 SSIM: 0.6989 |
| image-super-resolution-on-general100-4x | FxSR-PD t=0.8 | DISTS: 0.0831 LPIPS: 0.0784 LRPSNR: 49.82 NIQE: 4.54 PSNR: 28.44 SSIM: 0.8229 |
| image-super-resolution-on-general100-4x | FxSR-PD t=0.0 | DISTS: 0.1205 LPIPS: 0.1519 LRPSNR: 52.22 NIQE: 6.05 PSNR: 29.94 SSIM: 0.8629 |
| image-super-resolution-on-general100-8x | FxSR-PD t=0.8 | DISTS: 0.1716 LPIPS: 0.2058 LRPSNR: 41.36 NIQE: 5.46 PSNR: 24 SSIM: 0.6534 |
| image-super-resolution-on-general100-8x | FxSR-PD t=0.0 | DISTS: 0.2134 LPIPS: 0.2924 LRPSNR: 44.28 NIQE: 6.09 PSNR: 25.42 SSIM: 0.7097 |