Dudhane Akshay ; Zamir Syed Waqas ; Khan Salman ; Khan Fahad Shahbaz ; Yang Ming-Hsuan

Abstract

Modern handheld devices can acquire burst image sequence in a quicksuccession. However, the individual acquired frames suffer from multipledegradations and are misaligned due to camera shake and object motions. Thegoal of Burst Image Restoration is to effectively combine complimentary cuesacross multiple burst frames to generate high-quality outputs. Towards thisgoal, we develop a novel approach by solely focusing on the effectiveinformation exchange between burst frames, such that the degradations getfiltered out while the actual scene details are preserved and enhanced. Ourcentral idea is to create a set of pseudo-burst features that combinecomplementary information from all the input burst frames to seamlesslyexchange information. However, the pseudo-burst cannot be successfully createdunless the individual burst frames are properly aligned to discount inter-framemovements. Therefore, our approach initially extracts pre-processed featuresfrom each burst frame and matches them using an edge-boosting burst alignmentmodule. The pseudo-burst features are then created and enriched usingmulti-scale contextual information. Our final step is to adaptively aggregateinformation from the pseudo-burst features to progressively increase resolutionin multiple stages while merging the pseudo-burst features. In comparison toexisting works that usually follow a late fusion scheme with single-stageupsampling, our approach performs favorably, delivering state-of-the-artperformance on burst superresolution, burst low-light image enhancement, andburst denoising tasks. The source code and pre-trained models are available at\url{https://github.com/akshaydudhane16/BIPNet}.