Goswami Dipam ; Schuster René ; van de Weijer Joost ; Stricker Didier

Abstract

In class-incremental semantic segmentation (CISS), deep learningarchitectures suffer from the critical problems of catastrophic forgetting andsemantic background shift. Although recent works focused on these issues,existing classifier initialization methods do not address the background shiftproblem and assign the same initialization weights to both background and newforeground class classifiers. We propose to address the background shift with anovel classifier initialization method which employs gradient-based attributionto identify the most relevant weights for new classes from the classifier'sweights for the previous background and transfers these weights to the newclassifier. This warm-start weight initialization provides a general solutionapplicable to several CISS methods. Furthermore, it accelerates learning of newclasses while mitigating forgetting. Our experiments demonstrate significantimprovement in mIoU compared to the state-of-the-art CISS methods on thePascal-VOC 2012, ADE20K and Cityscapes datasets.