Jha Debesh ; Tomar Nikhil Kumar ; Ali Sharib ; Riegler Michael A. ; Johansen Håvard D. ; Johansen Dag ; de Lange Thomas ; Halvorsen Pål

Abstract

Deep learning in gastrointestinal endoscopy can assist to improve clinicalperformance and be helpful to assess lesions more accurately. To this extent,semantic segmentation methods that can perform automated real-time delineationof a region-of-interest, e.g., boundary identification of cancer orprecancerous lesions, can benefit both diagnosis and interventions. However,accurate and real-time segmentation of endoscopic images is extremelychallenging due to its high operator dependence and high-definition imagequality. To utilize automated methods in clinical settings, it is crucial todesign lightweight models with low latency such that they can be integratedwith low-end endoscope hardware devices. In this work, we propose NanoNet, anovel architecture for the segmentation of video capsule endoscopy andcolonoscopy images. Our proposed architecture allows real-time performance andhas higher segmentation accuracy compared to other more complex ones. We usevideo capsule endoscopy and standard colonoscopy datasets with polyps, and adataset consisting of endoscopy biopsies and surgical instruments, to evaluatethe effectiveness of our approach. Our experiments demonstrate the increasedperformance of our architecture in terms of a trade-off between modelcomplexity, speed, model parameters, and metric performances. Moreover, theresulting model size is relatively tiny, with only nearly 36,000 parameterscompared to traditional deep learning approaches having millions of parameters.