Bhat Goutam ; Danelljan Martin ; Yu Fisher ; Van Gool Luc ; Timofte Radu

Abstract

We propose a deep reparametrization of the maximum a posteriori formulationcommonly employed in multi-frame image restoration tasks. Our approach isderived by introducing a learned error metric and a latent representation ofthe target image, which transforms the MAP objective to a deep feature space.The deep reparametrization allows us to directly model the image formationprocess in the latent space, and to integrate learned image priors into theprediction. Our approach thereby leverages the advantages of deep learning,while also benefiting from the principled multi-frame fusion provided by theclassical MAP formulation. We validate our approach through comprehensiveexperiments on burst denoising and burst super-resolution datasets. Ourapproach sets a new state-of-the-art for both tasks, demonstrating thegenerality and effectiveness of the proposed formulation.