3 个月前

USB:通用尺度目标检测基准

USB:通用尺度目标检测基准

摘要

基准测试(如 COCO)在目标检测领域中发挥着至关重要的作用。然而,现有的基准测试在尺度变化方面仍显不足,且其评估协议难以实现公平比较。为此,本文提出通用尺度目标检测基准(Universal-Scale object detection Benchmark,简称 USB)。USB 通过整合 COCO 数据集与近期提出的 Waymo Open Dataset 及 Manga109-s 数据集,实现了对象尺度和图像领域上的多样化。为促进公平比较与包容性研究,我们设计了一套训练与评估协议。该协议采用类似体育竞赛中“权重级别”的多级划分机制,对训练轮次(epochs)和评估图像分辨率进行分级,并具备类似通用串行总线(USB)的向后兼容特性,确保不同训练协议之间的互操作性。具体而言,我们要求参赛者不仅报告在较高协议(更长训练周期)下的结果,也需提供在较低协议(较短训练周期)下的表现。基于所提出的基准与协议,我们使用 15 种方法开展了大量实验,揭示了现有以 COCO 为中心的方法在泛化能力上的固有缺陷。相关代码已开源,地址为:https://github.com/shinya7y/UniverseNet。

代码仓库

shinya7y/UniverseNet
官方
pytorch
GitHub 中提及

基准测试

基准方法指标
object-detection-on-cocoUniverseNet-20.08 (Res2Net-50, DCN, single-scale)
AP50: 67.5
AP75: 53.0
APL: 61.1
APM: 52.3
APS: 30.1
box mAP: 48.8
object-detection-on-cocoUniverseNet-20.08d (Res2Net-101, DCN, multi-scale)
AP50: 71.6
AP75: 59.9
APL: 67.4
APM: 57.2
APS: 35.8
box mAP: 54.1
object-detection-on-cocoUniverseNet-20.08d (Res2Net-101, DCN, single-scale)
AP50: 70.0
AP75: 55.8
APL: 64.9
APM: 55.3
APS: 31.7
box mAP: 51.3
object-detection-on-coco-minivalUniverseNet-20.08d (Res2Net-101, DCN, single-scale)
AP50: 69.5
AP75: 55.4
APL: 65.8
APM: 55.5
APS: 33.5
box AP: 50.9
object-detection-on-coco-minivalUniverseNet-20.08d (Res2Net-101, DCN, multi-scale)
AP50: 70.8
AP75: 58.9
APL: 68.1
APM: 57.5
APS: 36.9
box AP: 53.5
object-detection-on-coco-minivalUniverseNet-20.08 (Res2Net-50, DCN, single-scale)
AP50: 67.0
AP75: 52.6
APL: 62.7
APM: 52.7
APS: 30.6
box AP: 48.5
object-detection-on-coco-oUniverseNet (R2-101-DCN)
Effective Robustness: 1.86
object-detection-on-coco-oUniverseNet (R2-101-DCN)
Average mAP: 24.8
object-detection-on-manga109-s-15testCascade R-CNN
COCO-style AP: 67.6
object-detection-on-manga109-s-15testUniverseNet-20.08
COCO-style AP: 69.9
object-detection-on-manga109-s-15testATSS (ConvNeXt-T)
COCO-style AP: 67.4
object-detection-on-manga109-s-15testATSS (Swin-T)
COCO-style AP: 66.2
object-detection-on-manga109-s-15testDeformable DETR
COCO-style AP: 64.1
object-detection-on-manga109-s-15testSparse R-CNN
COCO-style AP: 63.1
object-detection-on-manga109-s-15testATSS+SEPC
COCO-style AP: 67.1
object-detection-on-manga109-s-15testATSS+DyHead
COCO-style AP: 67.9
object-detection-on-manga109-s-15testUniverseNet
COCO-style AP: 68.9
object-detection-on-manga109-s-15testDETR
COCO-style AP: 31.2
object-detection-on-manga109-s-15testFaster R-CNN
COCO-style AP: 65.8
object-detection-on-manga109-s-15testGFL
COCO-style AP: 67.3
object-detection-on-manga109-s-15testYOLOX-L
COCO-style AP: 70.2
object-detection-on-manga109-s-15testATSS
COCO-style AP: 66.5
object-detection-on-manga109-s-15testRetinaNet
COCO-style AP: 65.3
object-detection-on-usb-standard-usb-1-0YOLOX-L
mCAP: 49.6
object-detection-on-usb-standard-usb-1-0Cascade R-CNN
mCAP: 48.1
object-detection-on-usb-standard-usb-1-0GFL
mCAP: 47.7
object-detection-on-usb-standard-usb-1-0Sparse R-CNN
mCAP: 44.6
object-detection-on-usb-standard-usb-1-0UniverseNet
mCAP: 51.4
object-detection-on-usb-standard-usb-1-0Faster R-CNN
mCAP: 45.9
object-detection-on-usb-standard-usb-1-0UniverseNet-20.08
mCAP: 52.1
object-detection-on-usb-standard-usb-1-0ATSS
mCAP: 47.1
object-detection-on-usb-standard-usb-1-0ATSS+DyHead
mCAP: 49.4
object-detection-on-usb-standard-usb-1-0ATSS (Swin-T)
mCAP: 49.0
object-detection-on-usb-standard-usb-1-0RetinaNet
mCAP: 44.8
object-detection-on-usb-standard-usb-1-0ATSS (ConvNeXt-T)
mCAP: 50.4
object-detection-on-usb-standard-usb-1-0DETR
mCAP: 23.7
object-detection-on-waymo-2d-detection-all-nsUniverseNet
AP/L2: 67.42
object-detection-on-waymo-2d-detection-all-ns-1DETR
COCO-style AP: 17.8
object-detection-on-waymo-2d-detection-all-ns-1UniverseNet-20.08
COCO-style AP: 39.0
object-detection-on-waymo-2d-detection-all-ns-1YOLOX-L
COCO-style AP: 41.6
object-detection-on-waymo-2d-detection-all-ns-1ATSS+DyHead
COCO-style AP: 37.1
object-detection-on-waymo-2d-detection-all-ns-1GFL
COCO-style AP: 35.7
object-detection-on-waymo-2d-detection-all-ns-1Faster R-CNN
COCO-style AP: 34.5
object-detection-on-waymo-2d-detection-all-ns-1ATSS
COCO-style AP: 35.4
object-detection-on-waymo-2d-detection-all-ns-1UniverseNet
COCO-style AP: 38.6
object-detection-on-waymo-2d-detection-all-ns-1ATSS (Swin-T)
COCO-style AP: 37.2
object-detection-on-waymo-2d-detection-all-ns-1Sparse R-CNN
COCO-style AP: 32.8
object-detection-on-waymo-2d-detection-all-ns-1Cascade R-CNN
COCO-style AP: 36.4
object-detection-on-waymo-2d-detection-all-ns-1Deformable DETR
COCO-style AP: 32.7
object-detection-on-waymo-2d-detection-all-ns-1ATSS (ConvNeXt-T)
COCO-style AP: 38.3
object-detection-on-waymo-2d-detection-all-ns-1ATSS+SEPC
COCO-style AP: 35.0
object-detection-on-waymo-2d-detection-all-ns-1RetinaNet
COCO-style AP: 32.5

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程
即用型 GPU
最优价格
立即开始

Hyper Newsletters

订阅我们的最新资讯
我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新
邮件发送服务由 MailChimp 提供
USB:通用尺度目标检测基准 | 论文 | HyperAI超神经