Shen Xintian ; Zhang Jiangning ; Chen Jun ; Bai Shipeng ; Han Yue ; Wang Yabiao ; Wang Chengjie ; Liu Yong

Abstract

Image harmonization aims to solve the visual inconsistency problem incomposited images by adaptively adjusting the foreground pixels with thebackground as references. Existing methods employ local color transformation orregion matching between foreground and background, which neglects powerfulproximity prior and independently distinguishes fore-/back-ground as a wholepart for harmonization. As a result, they still show a limited performanceacross varied foreground objects and scenes. To address this issue, we proposea novel Global-aware Kernel Network (GKNet) to harmonize local regions withcomprehensive consideration of long-distance background references.Specifically, GKNet includes two parts, \ie, harmony kernel prediction andharmony kernel modulation branches. The former includes a Long-distanceReference Extractor (LRE) to obtain long-distance context and Kernel PredictionBlocks (KPB) to predict multi-level harmony kernels by fusing globalinformation with local features. To achieve this goal, a novel SelectiveCorrelation Fusion (SCF) module is proposed to better select relevantlong-distance background references for local harmonization. The latter employsthe predicted kernels to harmonize foreground regions with both local andglobal awareness. Abundant experiments demonstrate the superiority of ourmethod for image harmonization over state-of-the-art methods, \eg, achieving39.53dB PSNR that surpasses the best counterpart by +0.78dB $\uparrow$;decreasing fMSE/MSE by 11.5\%$\downarrow$/6.7\%$\downarrow$ compared with theSoTA method. Code will be available at\href{https://github.com/XintianShen/GKNet}{here}.