HyperAIHyperAI

Command Palette

Search for a command to run...

5 months ago

SwinIR: Image Restoration Using Swin Transformer

Liang Jingyun ; Cao Jiezhang ; Sun Guolei ; Zhang Kai ; Van Gool Luc ; Timofte Radu

SwinIR: Image Restoration Using Swin Transformer

Abstract

Image restoration is a long-standing low-level vision problem that aims torestore high-quality images from low-quality images (e.g., downscaled, noisyand compressed images). While state-of-the-art image restoration methods arebased on convolutional neural networks, few attempts have been made withTransformers which show impressive performance on high-level vision tasks. Inthis paper, we propose a strong baseline model SwinIR for image restorationbased on the Swin Transformer. SwinIR consists of three parts: shallow featureextraction, deep feature extraction and high-quality image reconstruction. Inparticular, the deep feature extraction module is composed of several residualSwin Transformer blocks (RSTB), each of which has several Swin Transformerlayers together with a residual connection. We conduct experiments on threerepresentative tasks: image super-resolution (including classical, lightweightand real-world image super-resolution), image denoising (including grayscaleand color image denoising) and JPEG compression artifact reduction.Experimental results demonstrate that SwinIR outperforms state-of-the-artmethods on different tasks by $\textbf{up to 0.14$\sim$0.45dB}$, while thetotal number of parameters can be reduced by $\textbf{up to 67%}$.

Code Repositories

mv-lab/swin2sr
pytorch
Mentioned in GitHub
rami0205/ngramswin
pytorch
Mentioned in GitHub
ayanglab/swinmr
pytorch
Mentioned in GitHub
skchen1993/SwinIR
pytorch
Mentioned in GitHub
jingyunliang/vrt
pytorch
Mentioned in GitHub
jingyunliang/swinir
Official
pytorch
Mentioned in GitHub
ayanglab/swinganmr
pytorch
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
color-image-denoising-on-kodak24-sigma50SwinIR
PSNR: 29.79
color-image-denoising-on-urban100-sigma10SwinIR
PSNR: 35.13
color-image-denoising-on-urban100-sigma15-1SwinIR
Average PSNR: 35.13
color-image-denoising-on-urban100-sigma25SwinIR
PSNR: 32.9
color-image-denoising-on-urban100-sigma50SwinIR
PSNR: 29.82
grayscale-image-denoising-on-bsd68-sigma15SwinIR
PSNR: 31.97
grayscale-image-denoising-on-urban100-sigma15SwinIR
PSNR: 33.70
grayscale-image-denoising-on-urban100-sigma25SwinIR
PSNR: 31.3
grayscale-image-denoising-on-urban100-sigma50SwinIR
PSNR: 27.98
image-super-resolution-on-manga109-4xSwinIR
PSNR: 32.22
SSIM: 0.9273
image-super-resolution-on-set14-4x-upscalingSwinIR
PSNR: 29.15
SSIM: 0.7958
image-super-resolution-on-urban100-4xSwinIR
PSNR: 27.45
SSIM: 0.8254
video-super-resolution-on-msu-super-1SwinIR + vvenc
BSQ-rate over ERQA: 6.624
BSQ-rate over LPIPS: 1.552
BSQ-rate over MS-SSIM: 5.758
BSQ-rate over PSNR: 8.971
BSQ-rate over Subjective Score: 1.35
BSQ-rate over VMAF: 0.887
video-super-resolution-on-msu-super-1SwinIR + aomenc
BSQ-rate over ERQA: 10.854
BSQ-rate over LPIPS: 4.566
BSQ-rate over MS-SSIM: 7.105
BSQ-rate over PSNR: 15.144
BSQ-rate over Subjective Score: 0.835
BSQ-rate over VMAF: 3.32
video-super-resolution-on-msu-super-1SwinIR + uavs3e
BSQ-rate over ERQA: 6.803
BSQ-rate over LPIPS: 1.671
BSQ-rate over MS-SSIM: 4.411
BSQ-rate over PSNR: 15.144
BSQ-rate over Subjective Score: 0.639
BSQ-rate over VMAF: 1.848
video-super-resolution-on-msu-super-1SwinIR + x265
BSQ-rate over ERQA: 1.575
BSQ-rate over LPIPS: 1.474
BSQ-rate over MS-SSIM: 4.641
BSQ-rate over PSNR: 8.13
BSQ-rate over Subjective Score: 0.346
BSQ-rate over VMAF: 1.304
video-super-resolution-on-msu-super-1SwinIR + x264
BSQ-rate over ERQA: 0.76
BSQ-rate over LPIPS: 0.559
BSQ-rate over MS-SSIM: 0.736
BSQ-rate over PSNR: 6.268
BSQ-rate over Subjective Score: 0.304
BSQ-rate over VMAF: 0.642
video-super-resolution-on-msu-video-upscalersSwinIR-Real-B
LPIPS: 0.183
PSNR: 28.86
SSIM: 0.830
video-super-resolution-on-msu-video-upscalersSwinIR-Real-S
LPIPS: 0.189
PSNR: 28.55
SSIM: 0.845
video-super-resolution-on-msu-vsr-benchmarkSwinIR
1 - LPIPS: 0.895
ERQAv1.0: 0.618
FPS: 0.407
PSNR: 25.12
QRCRv1.0: 0
SSIM: 0.782
Subjective score: 4.799

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
SwinIR: Image Restoration Using Swin Transformer | Papers | HyperAI