Tsai Yu-Ju ; Liu Yu-Lun ; Qi Lu ; Chan Kelvin C. K. ; Yang Ming-Hsuan

Abstract

Restoring facial details from low-quality (LQ) images has remained achallenging problem due to its ill-posedness induced by various degradations inthe wild. The existing codebook prior mitigates the ill-posedness by leveragingan autoencoder and learned codebook of high-quality (HQ) features, achievingremarkable quality. However, existing approaches in this paradigm frequentlydepend on a single encoder pre-trained on HQ data for restoring HQ images,disregarding the domain gap between LQ and HQ images. As a result, the encodingof LQ inputs may be insufficient, resulting in suboptimal performance. Totackle this problem, we propose a novel dual-branch framework named DAEFR. Ourmethod introduces an auxiliary LQ branch that extracts crucial information fromthe LQ inputs. Additionally, we incorporate association training to promoteeffective synergy between the two branches, enhancing code prediction andoutput quality. We evaluate the effectiveness of DAEFR on both synthetic andreal-world datasets, demonstrating its superior performance in restoring facialdetails. Project page: https://liagm.github.io/DAEFR/