Rossi Leonardo ; Bernuzzi Vittorio ; Fontanini Tomaso ; Bertozzi Massimo ; Prati Andrea

Abstract

Due to the limitations of current optical and sensor technologies and thehigh cost of updating them, the spectral and spatial resolution of satellitesmay not always meet desired requirements. For these reasons, Remote-SensingSingle-Image Super-Resolution (RS-SISR) techniques have gained significantinterest. In this paper, we propose Swin2-MoSE model, an enhanced version ofSwin2SR. Our model introduces MoE-SM, an enhanced Mixture-of-Experts (MoE) toreplace the Feed-Forward inside all Transformer block. MoE-SM is designed withSmart-Merger, and new layer for merging the output of individual experts, andwith a new way to split the work between experts, defining a new per-examplestrategy instead of the commonly used per-token one. Furthermore, we analyzehow positional encodings interact with each other, demonstrating thatper-channel bias and per-head bias can positively cooperate. Finally, wepropose to use a combination of Normalized-Cross-Correlation (NCC) andStructural Similarity Index Measure (SSIM) losses, to avoid typical MSE losslimitations. Experimental results demonstrate that Swin2-MoSE outperforms anySwin derived models by up to 0.377 - 0.958 dB (PSNR) on task of 2x, 3x and 4xresolution-upscaling (Sen2Venus and OLI2MSI datasets). It also outperforms SOTAmodels by a good margin, proving to be competitive and with excellentpotential, especially for complex tasks. Additionally, an analysis ofcomputational costs is also performed. Finally, we show the efficacy ofSwin2-MoSE, applying it to a semantic segmentation task (SeasoNet dataset).Code and pretrained are available onhttps://github.com/IMPLabUniPr/swin2-mose/tree/official_code