Abstract

We propose a technique for producing "visual explanations" for decisions froma large class of CNN-based models, making them more transparent. Our approach -Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients ofany target concept, flowing into the final convolutional layer to produce acoarse localization map highlighting important regions in the image forpredicting the concept. Grad-CAM is applicable to a wide variety of CNNmodel-families: (1) CNNs with fully-connected layers, (2) CNNs used forstructured outputs, (3) CNNs used in tasks with multimodal inputs orreinforcement learning, without any architectural changes or re-training. Wecombine Grad-CAM with fine-grained visualizations to create a high-resolutionclass-discriminative visualization and apply it to off-the-shelf imageclassification, captioning, and visual question answering (VQA) models,including ResNet-based architectures. In the context of image classificationmodels, our visualizations (a) lend insights into their failure modes, (b) arerobust to adversarial images, (c) outperform previous methods on localization,(d) are more faithful to the underlying model and (e) help achievegeneralization by identifying dataset bias. For captioning and VQA, we showthat even non-attention based models can localize inputs. We devise a way toidentify important neurons through Grad-CAM and combine it with neuron names toprovide textual explanations for model decisions. Finally, we design andconduct human studies to measure if Grad-CAM helps users establish appropriatetrust in predictions from models and show that Grad-CAM helps untrained userssuccessfully discern a 'stronger' nodel from a 'weaker' one even when both makeidentical predictions. Our code is available athttps://github.com/ramprs/grad-cam/, along with a demo athttp://gradcam.cloudcv.org, and a video at youtu.be/COjUB9Izk6E.