Janjušević Nikola ; Khalilian-Gourtani Amirhossein ; Flinker Adeen ; Wang Yao

Abstract

Nonlocal self-similarity within natural images has become an increasinglypopular prior in deep-learning models. Despite their successful imagerestoration performance, such models remain largely uninterpretable due totheir black-box construction. Our previous studies have shown thatinterpretable construction of a fully convolutional denoiser (CDLNet), withperformance on par with state-of-the-art black-box counterparts, is achievableby unrolling a dictionary learning algorithm. In this manuscript, we seek aninterpretable construction of a convolutional network with a nonlocalself-similarity prior that performs on par with black-box nonlocal models. Weshow that such an architecture can be effectively achieved by upgrading the$\ell 1$ sparsity prior of CDLNet to a weighted group-sparsity prior. From thisformulation, we propose a novel sliding-window nonlocal operation, enabled bysparse array arithmetic. In addition to competitive performance with black-boxnonlocal DNNs, we demonstrate the proposed sliding-window sparse attentionenables inference speeds greater than an order of magnitude faster than itscompetitors.