Lin Xinqi ; He Jingwen ; Chen Ziyan ; Lyu Zhaoyang ; Dai Bo ; Yu Fanghua ; Ouyang Wanli ; Qiao Yu ; Dong Chao

Abstract

We present DiffBIR, a general restoration pipeline that could handledifferent blind image restoration tasks in a unified framework. DiffBIRdecouples blind image restoration problem into two stages: 1) degradationremoval: removing image-independent content; 2) information regeneration:generating the lost image content. Each stage is developed independently butthey work seamlessly in a cascaded manner. In the first stage, we userestoration modules to remove degradations and obtain high-fidelity restoredresults. For the second stage, we propose IRControlNet that leverages thegenerative ability of latent diffusion models to generate realistic details.Specifically, IRControlNet is trained based on specially produced conditionimages without distracting noisy content for stable generation performance.Moreover, we design a region-adaptive restoration guidance that can modify thedenoising process during inference without model re-training, allowing users tobalance realness and fidelity through a tunable guidance scale. Extensiveexperiments have demonstrated DiffBIR's superiority over state-of-the-artapproaches for blind image super-resolution, blind face restoration and blindimage denoising tasks on both synthetic and real-world datasets. The code isavailable at https://github.com/XPixelGroup/DiffBIR.