Zhang Shu ; Xu Jincheng ; Chen Yu-Chun ; Ma Jiechao ; Li Zihao ; Wang Yizhou ; Yu Yizhou

Abstract

Universal lesion detection from computed tomography (CT) slices is importantfor comprehensive disease screening. Since each lesion can locate in multipleadjacent slices, 3D context modeling is of great significance for developingautomated lesion detection algorithms. In this work, we propose a ModifiedPseudo-3D Feature Pyramid Network (MP3D FPN) that leverages depthwise separableconvolutional filters and a group transform module (GTM) to efficiently extract3D context enhanced 2D features for universal lesion detection in CT slices. Tofacilitate faster convergence, a novel 3D network pre-training method isderived using solely large-scale 2D object detection dataset in the naturalimage domain. We demonstrate that with the novel pre-training method, theproposed MP3D FPN achieves state-of-the-art detection performance on theDeepLesion dataset (3.48% absolute improvement in the sensitivity of FPs@0.5),significantly surpassing the baseline method by up to 6.06% (in MAP@0.5) whichadopts 2D convolution for 3D context modeling. Moreover, the proposed 3Dpre-trained weights can potentially be used to boost the performance of other3D medical image analysis tasks.