3 个月前

通道分区窗口化注意力与频率学习用于单图像超分辨率

通道分区窗口化注意力与频率学习用于单图像超分辨率

摘要

近年来,基于窗口的注意力机制在计算机视觉任务中展现出巨大潜力,尤其在单图像超分辨率(Single Image Super-Resolution, SISR)任务中表现突出。然而,此类方法在捕捉远距离特征之间的长程依赖关系方面仍存在局限。此外,我们发现,仅在空间域进行学习无法有效表达图像的频率信息,而频率特性在SISR任务中至关重要。为解决上述问题,本文提出一种新型通道分割注意力变换器(Channel-Partitioned Attention Transformer, CPAT),通过沿特征图的高度和宽度方向逐次扩展窗口,更有效地建模长程依赖关系。同时,我们设计了一种新颖的空间-频率交互模块(Spatial-Frequency Interaction Module, SFIM),该模块融合空间域与频率域的信息,从而从特征图中提取更全面的表征,不仅包含频率成分信息,还显著扩展了模型在整个图像上的感受野。实验结果表明,所提出的模块与整体架构具有显著有效性。特别是在Urban100数据集上进行x2超分辨率重建时,CPAT性能超越当前最优方法,最高提升达0.31dB。

基准测试

基准方法指标
image-super-resolution-on-bsd100-2x-upscalingCPAT
PSNR: 32.64
SSIM: 0.9056
image-super-resolution-on-bsd100-2x-upscalingCPAT+
PSNR: 32.66
SSIM: 0.9058
image-super-resolution-on-bsd100-3x-upscalingCPAT
PSNR: 29.56
SSIM: 0.8174
image-super-resolution-on-bsd100-3x-upscalingCPAT+
PSNR: 29.59
SSIM: 0.8177
image-super-resolution-on-bsd100-4x-upscalingCPAT+
PSNR: 28.06
SSIM: 0.7532
image-super-resolution-on-bsd100-4x-upscalingCPAT
PSNR: 28.04
SSIM: 0.7527
image-super-resolution-on-manga109-2xCPAT+
PSNR: 40.59
SSIM: 0.9816
image-super-resolution-on-manga109-2xCPAT
PSNR: 40.48
SSIM: 0.9814
image-super-resolution-on-manga109-3xCPAT
PSNR: 35.66
SSIM: 0.9559
image-super-resolution-on-manga109-3xCPAT+
PSNR: 35.77
SSIM: 0.9563
image-super-resolution-on-manga109-4xCPAT+
PSNR: 32.85
SSIM: 0.9318
image-super-resolution-on-set14-2x-upscalingCPAT
PSNR: 34.91
SSIM: 0.9277
image-super-resolution-on-set14-2x-upscalingCPAT+
PSNR: 34.97
SSIM: 0.9280
image-super-resolution-on-set14-3x-upscalingCPAT
PSNR: 31.15
SSIM: 0.8557
image-super-resolution-on-set14-3x-upscalingCPAT+
PSNR: 31.19
SSIM: 0.8559
image-super-resolution-on-set14-4x-upscalingCPAT
PSNR: 29.34
SSIM: 0.7991
image-super-resolution-on-set14-4x-upscalingCPAT+
PSNR: 29.36
SSIM: 0.7996
image-super-resolution-on-set5-2x-upscalingCPAT
PSNR: 38.68
SSIM: 0.9633
image-super-resolution-on-set5-2x-upscalingCPAT+
PSNR: 38.72
SSIM: 0.9635
image-super-resolution-on-set5-3x-upscalingCPAT
PSNR: 35.16
SSIM: 0.9334
image-super-resolution-on-set5-3x-upscalingCPAT+
PSNR: 35.19
SSIM: 0.9335
image-super-resolution-on-set5-4x-upscalingCPAT+
PSNR: 33.24
SSIM: 0.9071
image-super-resolution-on-set5-4x-upscalingCPAT
PSNR: 33.19
SSIM: 0.9069
image-super-resolution-on-urban100-2xCPAT+
PSNR: 34.89
SSIM: 0.9487
image-super-resolution-on-urban100-2xCPAT
PSNR: 34.76
SSIM: 0.9481
image-super-resolution-on-urban100-3xCPAT+
PSNR: 30.63
SSIM: 0.8934
image-super-resolution-on-urban100-3xCPAT
PSNR: 30.52
SSIM: 0.8923
image-super-resolution-on-urban100-4xCPAT
PSNR: 28.22
SSIM: 0.8408
image-super-resolution-on-urban100-4xCPAT+
PSNR: 28.33
SSIM: 0.8425

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程
即用型 GPU
最优价格
立即开始

Hyper Newsletters

订阅我们的最新资讯
我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新
邮件发送服务由 MailChimp 提供
通道分区窗口化注意力与频率学习用于单图像超分辨率 | 论文 | HyperAI超神经