Li Yawei ; Ren Bin ; Liang Jingyun ; Ranjan Rakesh ; Liu Mengyuan ; Sebe Nicu ; Yang Ming-Hsuan ; Benini Luca

Abstract

While vision transformers show promise in numerous image restoration (IR)tasks, the challenge remains in efficiently generalizing and scaling up a modelfor multiple IR tasks. To strike a balance between efficiency and modelcapacity for a generalized transformer-based IR method, we propose ahierarchical information flow mechanism for image restoration, dubbed Hi-IR,which progressively propagates information among pixels in a bottom-up manner.Hi-IR constructs a hierarchical information tree representing the degradedimage across three levels. Each level encapsulates different types ofinformation, with higher levels encompassing broader objects and concepts andlower levels focusing on local details. Moreover, the hierarchical treearchitecture removes long-range self-attention, improves the computationalefficiency and memory utilization, thus preparing it for effective modelscaling. Based on that, we explore model scaling to improve our method'scapabilities, which is expected to positively impact IR in large-scale trainingsettings. Extensive experimental results show that Hi-IR achievesstate-of-the-art performance in seven common image restoration tasks, affirmingits effectiveness and generalizability.