Command Palette
Search for a command to run...
Liang Jingyun ; Cao Jiezhang ; Sun Guolei ; Zhang Kai ; Van Gool Luc ; Timofte Radu

Abstract
Image restoration is a long-standing low-level vision problem that aims torestore high-quality images from low-quality images (e.g., downscaled, noisyand compressed images). While state-of-the-art image restoration methods arebased on convolutional neural networks, few attempts have been made withTransformers which show impressive performance on high-level vision tasks. Inthis paper, we propose a strong baseline model SwinIR for image restorationbased on the Swin Transformer. SwinIR consists of three parts: shallow featureextraction, deep feature extraction and high-quality image reconstruction. Inparticular, the deep feature extraction module is composed of several residualSwin Transformer blocks (RSTB), each of which has several Swin Transformerlayers together with a residual connection. We conduct experiments on threerepresentative tasks: image super-resolution (including classical, lightweightand real-world image super-resolution), image denoising (including grayscaleand color image denoising) and JPEG compression artifact reduction.Experimental results demonstrate that SwinIR outperforms state-of-the-artmethods on different tasks by $\textbf{up to 0.14$\sim$0.45dB}$, while thetotal number of parameters can be reduced by $\textbf{up to 67%}$.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| color-image-denoising-on-kodak24-sigma50 | SwinIR | PSNR: 29.79 |
| color-image-denoising-on-urban100-sigma10 | SwinIR | PSNR: 35.13 |
| color-image-denoising-on-urban100-sigma15-1 | SwinIR | Average PSNR: 35.13 |
| color-image-denoising-on-urban100-sigma25 | SwinIR | PSNR: 32.9 |
| color-image-denoising-on-urban100-sigma50 | SwinIR | PSNR: 29.82 |
| grayscale-image-denoising-on-bsd68-sigma15 | SwinIR | PSNR: 31.97 |
| grayscale-image-denoising-on-urban100-sigma15 | SwinIR | PSNR: 33.70 |
| grayscale-image-denoising-on-urban100-sigma25 | SwinIR | PSNR: 31.3 |
| grayscale-image-denoising-on-urban100-sigma50 | SwinIR | PSNR: 27.98 |
| image-super-resolution-on-manga109-4x | SwinIR | PSNR: 32.22 SSIM: 0.9273 |
| image-super-resolution-on-set14-4x-upscaling | SwinIR | PSNR: 29.15 SSIM: 0.7958 |
| image-super-resolution-on-urban100-4x | SwinIR | PSNR: 27.45 SSIM: 0.8254 |
| video-super-resolution-on-msu-super-1 | SwinIR + vvenc | BSQ-rate over ERQA: 6.624 BSQ-rate over LPIPS: 1.552 BSQ-rate over MS-SSIM: 5.758 BSQ-rate over PSNR: 8.971 BSQ-rate over Subjective Score: 1.35 BSQ-rate over VMAF: 0.887 |
| video-super-resolution-on-msu-super-1 | SwinIR + aomenc | BSQ-rate over ERQA: 10.854 BSQ-rate over LPIPS: 4.566 BSQ-rate over MS-SSIM: 7.105 BSQ-rate over PSNR: 15.144 BSQ-rate over Subjective Score: 0.835 BSQ-rate over VMAF: 3.32 |
| video-super-resolution-on-msu-super-1 | SwinIR + uavs3e | BSQ-rate over ERQA: 6.803 BSQ-rate over LPIPS: 1.671 BSQ-rate over MS-SSIM: 4.411 BSQ-rate over PSNR: 15.144 BSQ-rate over Subjective Score: 0.639 BSQ-rate over VMAF: 1.848 |
| video-super-resolution-on-msu-super-1 | SwinIR + x265 | BSQ-rate over ERQA: 1.575 BSQ-rate over LPIPS: 1.474 BSQ-rate over MS-SSIM: 4.641 BSQ-rate over PSNR: 8.13 BSQ-rate over Subjective Score: 0.346 BSQ-rate over VMAF: 1.304 |
| video-super-resolution-on-msu-super-1 | SwinIR + x264 | BSQ-rate over ERQA: 0.76 BSQ-rate over LPIPS: 0.559 BSQ-rate over MS-SSIM: 0.736 BSQ-rate over PSNR: 6.268 BSQ-rate over Subjective Score: 0.304 BSQ-rate over VMAF: 0.642 |
| video-super-resolution-on-msu-video-upscalers | SwinIR-Real-B | LPIPS: 0.183 PSNR: 28.86 SSIM: 0.830 |
| video-super-resolution-on-msu-video-upscalers | SwinIR-Real-S | LPIPS: 0.189 PSNR: 28.55 SSIM: 0.845 |
| video-super-resolution-on-msu-vsr-benchmark | SwinIR | 1 - LPIPS: 0.895 ERQAv1.0: 0.618 FPS: 0.407 PSNR: 25.12 QRCRv1.0: 0 SSIM: 0.782 Subjective score: 4.799 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.