
摘要
现有的大多数图像复原(Image Restoration, IR)模型均为任务特定型,难以泛化至不同的退化算子。在本工作中,我们提出了一种新型的零样本框架——去噪扩散零空间模型(Denoising Diffusion Null-Space Model, DDNM),可适用于任意线性图像复原任务,包括但不限于图像超分辨率、着色、图像修复、压缩感知以及去模糊。DDNM仅需一个预训练的现成扩散模型作为生成先验,无需任何额外训练或网络结构修改。通过在逆向扩散过程中仅优化零空间(null-space)内容,即可生成满足数据一致性与真实感的多样化复原结果。为进一步提升性能,我们还提出了增强且更鲁棒的版本——DDNM+,以支持含噪复原,并显著提升在高难度任务中的复原质量。在多个图像复原任务上的实验结果表明,DDNM优于现有的其他先进零样本复原方法。此外,我们还展示了DDNM+在复杂真实场景应用中的有效性,例如老旧照片的修复。
代码仓库
andreamazzitelli/ProjectNN
GitHub 中提及
ipc-lab/deepjscc-diffusion
pytorch
GitHub 中提及
xypeng9903/k-diffusion-inverse-problems
pytorch
GitHub 中提及
wyhuai/ddnm
官方
pytorch
GitHub 中提及
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| image-deblurring-on-celeba | A+y | FID: 54.31 PSNR: 18.85 SSIM: 0.741 |
| image-deblurring-on-celeba | DDRM | FID: 6.24 PSNR: 43.07 SSIM: 0.993 |
| image-deblurring-on-celeba | DDNM | FID: 1.41 PSNR: 46.72 SSIM: 0.996 |
| image-deblurring-on-imagenet | DDRM | FID: 1.48 PSNR: 43.01 SSIM: 0.992 |
| image-deblurring-on-imagenet | A+y | FID: 55.42 PSNR: 18.56 SSIM: 0.6616 |
| image-deblurring-on-imagenet | DDNM | FID: 1.15 PSNR: 44.93 SSIM: 0.994 |
| image-inpainting-on-celeba | A+y | FID: 181.56 PSNR: 15.57 SSIM: 0.809 |
| image-inpainting-on-celeba | RePaint | FID: 14.19 PSNR: 35.2 SSIM: 0.981 |
| image-inpainting-on-celeba | DDNM | FID: 4.54 PSNR: 35.64 SSIM: 0.982 |
| image-inpainting-on-celeba | DDRM | FID: 12.53 PSNR: 34.79 SSIM: 0.978 |
| image-inpainting-on-imagenet | DDRM | FID: 4.82 PSNR: 31.73 SSIM: 0.966 |
| image-inpainting-on-imagenet | A+y | FID: 72.71 PSNR: 14.52 SSIM: 0.799 |
| image-inpainting-on-imagenet | RePaint | FID: 12.31 PSNR: 31.87 SSIM: 0.963 |
| image-inpainting-on-imagenet | DDNM | FID: 3.89 PSNR: 32.06 SSIM: 0.968 |
| image-super-resolution-on-celeba | A+y | FID: 103.3 PSNR: 27.27 SSIM: 0.782 |
| image-super-resolution-on-celeba | DDNM | FID: 22.27 PSNR: 31.63 SSIM: 0.945 |
| image-super-resolution-on-celeba | PULSE | FID: 40.33 PSNR: 22.74 SSIM: 0.623 |
| image-super-resolution-on-celeba | ILVR | FID: 29.82 PSNR: 31.59 SSIM: 0.945 |
| image-super-resolution-on-celeba | DDRM | FID: 31.04 PSNR: 31.63 SSIM: 0.945 |
| image-super-resolution-on-imagenet | ILVR | FID: 43.66 PSNR: 27.4 SSIM: 0.87 |
| image-super-resolution-on-imagenet | A+y | FID: 134.4 PSNR: 24.26 SSIM: 0.684 |
| image-super-resolution-on-imagenet | DDNM | FID: 39.26 PSNR: 27.46 SSIM: 0.87 |
| image-super-resolution-on-imagenet | DGP | FID: 64.34 PSNR: 23.18 SSIM: 0.798 |
| image-super-resolution-on-imagenet | DDRM | FID: 43.15 PSNR: 27.38 SSIM: 0.869 |