Tang Xiaole ; Gu Xiang ; He Xiaoyi ; Hu Xin ; Sun Jian

Abstract

All-in-one image restoration has emerged as a practical and promisinglow-level vision task for real-world applications. In this context, the keyissue lies in how to deal with different types of degraded imagessimultaneously. In this work, we present a Degradation-AwareResidual-Conditioned Optimal Transport (DA-RCOT) approach that models(all-in-one) image restoration as an optimal transport (OT) problem forunpaired and paired settings, introducing the transport residual as adegradation-specific cue for both the transport cost and the transport map.Specifically, we formalize image restoration with a residual-guided OTobjective by exploiting the degradation-specific patterns of the Fourierresidual in the transport cost. More crucially, we design the transport map forrestoration as a two-pass DA-RCOT map, in which the transport residual iscomputed in the first pass and then encoded as multi-scale residual embeddingsto condition the second-pass restoration. This conditioning process injectsintrinsic degradation knowledge (e.g., degradation type and level) andstructural information from the multi-scale residual embeddings into the OTmap, which thereby can dynamically adjust its behaviors for all-in-onerestoration. Extensive experiments across five degradations demonstrate thefavorable performance of DA-RCOT as compared to state-of-the-art methods, interms of distortion measures, perceptual quality, and image structurepreservation. Notably, DA-RCOT delivers superior adaptability to real-worldscenarios even with multiple degradations and shows distinctive robustness toboth degradation levels and the number of degradations.