Nan Zhixiong Li Xianghong Dai Jifeng Xiang Tao

Abstract

Based on analyzing the character of cascaded decoder architecture commonlyadopted in existing DETR-like models, this paper proposes a new decoderarchitecture. The cascaded decoder architecture constrains object queries toupdate in the cascaded direction, only enabling object queries to learnrelatively-limited information from image features. However, the challenges forobject detection in natural scenes (e.g., extremely-small, heavily-occluded,and confusingly mixed with the background) require an object detection model tofully utilize image features, which motivates us to propose a new decoderarchitecture with the parallel Multi-time Inquiries (MI) mechanism. MI enablesobject queries to learn more comprehensive information, and our MI based model,MI-DETR, outperforms all existing DETR-like models on COCO benchmark underdifferent backbones and training epochs, achieving +2.3 AP and +0.6 APimprovements compared to the most representative model DINO and SOTA modelRelation-DETR under ResNet-50 backbone. In addition, a series of diagnostic andvisualization experiments demonstrate the effectiveness, rationality, andinterpretability of MI.