Chu Xiaojie ; Chen Liangyu ; Chen Chengpeng ; Lu Xin

Abstract

Global operations, such as global average pooling, are widely used intop-performance image restorers. They aggregate global information from inputfeatures along entire spatial dimensions but behave differently during trainingand inference in image restoration tasks: they are based on different regions,namely the cropped patches (from images) and the full-resolution images. Thispaper revisits global information aggregation and finds that the image-basedfeatures during inference have a different distribution than the patch-basedfeatures during training. This train-test inconsistency negatively impacts theperformance of models, which is severely overlooked by previous works. Toreduce the inconsistency and improve test-time performance, we propose a simplemethod called Test-time Local Converter (TLC). Our TLC converts globaloperations to local ones only during inference so that they aggregate featureswithin local spatial regions rather than the entire large images. The proposedmethod can be applied to various global modules (e.g., normalization, channeland spatial attention) with negligible costs. Without the need for anyfine-tuning, TLC improves state-of-the-art results on several image restorationtasks, including single-image motion deblurring, video deblurring, defocusdeblurring, and image denoising. In particular, with TLC, our Restormer-Localimproves the state-of-the-art result in single image deblurring from 32.92 dBto 33.57 dB on GoPro dataset. The code is available athttps://github.com/megvii-research/tlc.