Kang EungGu ; Lee Byeonghun ; Im Sunghoon ; Jin Kyong Hwan

Abstract

Multi frame super-resolution(MFSR) achieves higher performance than singleimage super-resolution (SISR), because MFSR leverages abundant information frommultiple frames. Recent MFSR approaches adapt the deformable convolutionnetwork (DCN) to align the frames. However, the existing MFSR suffers frommisalignments between the reference and source frames due to the limitations ofDCN, such as small receptive fields and the predefined number of kernels. Fromthese problems, existing MFSR approaches struggle to represent high-frequencyinformation. To this end, we propose Deep Burst Multi-scale SR using FourierSpace with Optical Flow (BurstM). The proposed method estimates the opticalflow offset for accurate alignment and predicts the continuous Fouriercoefficient of each frame for representing high-frequency textures. Inaddition, we have enhanced the network flexibility by supporting varioussuper-resolution (SR) scale factors with the unimodel. We demonstrate that ourmethod has the highest performance and flexibility than the existing MFSRmethods. Our source code is available at https://github.com/Egkang-Luis/burstm