Chan Kelvin C. K. ; Zhou Shangchen ; Xu Xiangyu ; Loy Chen Change

Abstract

A recurrent structure is a popular framework choice for the task of videosuper-resolution. The state-of-the-art method BasicVSR adopts bidirectionalpropagation with feature alignment to effectively exploit information from theentire input video. In this study, we redesign BasicVSR by proposingsecond-order grid propagation and flow-guided deformable alignment. We showthat by empowering the recurrent framework with the enhanced propagation andalignment, one can exploit spatiotemporal information across misaligned videoframes more effectively. The new components lead to an improved performanceunder a similar computational constraint. In particular, our model BasicVSR++surpasses BasicVSR by 0.82 dB in PSNR with similar number of parameters. Inaddition to video super-resolution, BasicVSR++ generalizes well to other videorestoration tasks such as compressed video enhancement. In NTIRE 2021,BasicVSR++ obtains three champions and one runner-up in the VideoSuper-Resolution and Compressed Video Enhancement Challenges. Codes and modelswill be released to MMEditing.