Meng Yue ; Lu Yongxi ; Raj Aman ; Sunarjo Samuel ; Guo Rui ; Javidi Tara ; Bansal Gaurav ; Bharadia Dinesh

Abstract

Unsupervised learning for geometric perception (depth, optical flow, etc.) isof great interest to autonomous systems. Recent works on unsupervised learninghave made considerable progress on perceiving geometry; however, they usuallyignore the coherence of objects and perform poorly under scenarios with darkand noisy environments. In contrast, supervised learning algorithms, which arerobust, require large labeled geometric dataset. This paper introduces SIGNet,a novel framework that provides robust geometry perception without requiringgeometrically informative labels. Specifically, SIGNet integrates semanticinformation to make depth and flow predictions consistent with objects androbust to low lighting conditions. SIGNet is shown to improve upon thestate-of-the-art unsupervised learning for depth prediction by 30% (in squaredrelative error). In particular, SIGNet improves the dynamic object classperformance by 39% in depth prediction and 29% in flow prediction. Our codewill be made available at https://github.com/mengyuest/SIGNet