Dolz Jose ; Gopinath Karthik ; Yuan Jing ; Lombaert Herve ; Desrosiers Christian ; Ayed Ismail Ben

Abstract

Recently, dense connections have attracted substantial attention in computervision because they facilitate gradient flow and implicit deep supervisionduring training. Particularly, DenseNet, which connects each layer to everyother layer in a feed-forward fashion, has shown impressive performances innatural image classification tasks. We propose HyperDenseNet, a 3D fullyconvolutional neural network that extends the definition of dense connectivityto multi-modal segmentation problems. Each imaging modality has a path, anddense connections occur not only between the pairs of layers within the samepath, but also between those across different paths. This contrasts with theexisting multi-modal CNN approaches, in which modeling several modalitiesrelies entirely on a single joint layer (or level of abstraction) for fusion,typically either at the input or at the output of the network. Therefore, theproposed network has total freedom to learn more complex combinations betweenthe modalities, within and in-between all the levels of abstraction, whichincreases significantly the learning representation. We report extensiveevaluations over two different and highly competitive multi-modal brain tissuesegmentation challenges, iSEG 2017 and MRBrainS 2013, with the former focusingon 6-month infant data and the latter on adult images. HyperDenseNet yieldedsignificant improvements over many state-of-the-art segmentation networks,ranking at the top on both benchmarks. We further provide a comprehensiveexperimental analysis of features re-use, which confirms the importance ofhyper-dense connections in multi-modal representation learning. Our code ispublicly available at https://www.github.com/josedolz/HyperDenseNet.