Cermelli Fabio ; Mancini Massimiliano ; Bulò Samuel Rota ; Ricci Elisa ; Caputo Barbara

Abstract

Despite their effectiveness in a wide range of tasks, deep architecturessuffer from some important limitations. In particular, they are vulnerable tocatastrophic forgetting, i.e. they perform poorly when they are required toupdate their model as new classes are available but the original training setis not retained. This paper addresses this problem in the context of semanticsegmentation. Current strategies fail on this task because they do not considera peculiar aspect of semantic segmentation: since each training step providesannotation only for a subset of all possible classes, pixels of the backgroundclass (i.e. pixels that do not belong to any other classes) exhibit a semanticdistribution shift. In this work we revisit classical incremental learningmethods, proposing a new distillation-based framework which explicitly accountsfor this shift. Furthermore, we introduce a novel strategy to initializeclassifier's parameters, thus preventing biased predictions toward thebackground class. We demonstrate the effectiveness of our approach with anextensive evaluation on the Pascal-VOC 2012 and ADE20K datasets, significantlyoutperforming state of the art incremental learning methods.