Zhang Xinyu ; Huang Hefei ; Jia Xu ; Wang Dong ; Lu Huchuan

Abstract

The shutter strategy applied to the photo-shooting process has a significantinfluence on the quality of the captured photograph. An improper shutter maylead to a blurry image, video discontinuity, or rolling shutter artifact.Existing works try to provide an independent solution for each issue. In thiswork, we aim to re-expose the captured photo in post-processing to provide amore flexible way of addressing those issues within a unified framework.Specifically, we propose a neural network-based image re-exposure framework. Itconsists of an encoder for visual latent space construction, a re-exposuremodule for aggregating information to neural film with a desired shutterstrategy, and a decoder for 'developing' neural film into a desired image. Tocompensate for information confusion and missing frames, event streams, whichcan capture almost continuous brightness changes, are leveraged in computingvisual latent content. Both self-attention layers and cross-attention layersare employed in the re-exposure module to promote interaction between neuralfilm and visual latent content and information aggregation to neural film. Theproposed unified image re-exposure framework is evaluated on severalshutter-related image recovery tasks and performs favorably against independentstate-of-the-art methods.