Sarukkai Vishnu ; Jain Anirudh ; Uzkent Burak ; Ermon Stefano

Abstract

Satellite images hold great promise for continuous environmental monitoringand earth observation. Occlusions cast by clouds, however, can severely limitcoverage, making ground information extraction more difficult. Existingpipelines typically perform cloud removal with simple temporal composites andhand-crafted filters. In contrast, we cast the problem of cloud removal as aconditional image synthesis challenge, and we propose a trainablespatiotemporal generator network (STGAN) to remove clouds. We train our modelon a new large-scale spatiotemporal dataset that we construct, containing 97640image pairs covering all continents. We demonstrate experimentally that theproposed STGAN model outperforms standard models and can generate realisticcloud-free images with high PSNR and SSIM values across a variety ofatmospheric conditions, leading to improved performance in downstream taskssuch as land cover classification.