HyperAIHyperAI

Command Palette

Search for a command to run...

5 months ago

What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis

Baek Jeonghun ; Kim Geewook ; Lee Junyeop ; Park Sungrae ; Han Dongyoon ; Yun Sangdoo ; Oh Seong Joon ; Lee Hwalsuk

What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and
  Model Analysis

Abstract

Many new proposals for scene text recognition (STR) models have beenintroduced in recent years. While each claim to have pushed the boundary of thetechnology, a holistic and fair comparison has been largely missing in thefield due to the inconsistent choices of training and evaluation datasets. Thispaper addresses this difficulty with three major contributions. First, weexamine the inconsistencies of training and evaluation datasets, and theperformance gap results from inconsistencies. Second, we introduce a unifiedfour-stage STR framework that most existing STR models fit into. Using thisframework allows for the extensive evaluation of previously proposed STRmodules and the discovery of previously unexplored module combinations. Third,we analyze the module-wise contributions to performance in terms of accuracy,speed, and memory demand, under one consistent set of training and evaluationdatasets. Such analyses clean up the hindrance on the current comparisons tounderstand the performance gain of the existing modules.

Code Repositories

Luckygyana/Invo-AI
pytorch
Mentioned in GitHub
Eom-taeseon/CV_SceneTextRecognition
pytorch
Mentioned in GitHub
clovaai/deep-text-recognition-benchmark
Official
pytorch
Mentioned in GitHub
luozhilin99/cnnctc
mindspore
Mentioned in GitHub
2023-MindSpore-1/ms-code-58
mindspore
Mentioned in GitHub
dmitrijsk/attentionhtr
pytorch
Mentioned in GitHub
jakartaresearch/receipt-ocr
pytorch
Mentioned in GitHub
JoegameZhou/cnnctc
mindspore
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
scene-text-recognition-on-icdar-2003Baek et al.
Accuracy: 94.4
scene-text-recognition-on-icdar2013Baek et al.
Accuracy: 92.3
scene-text-recognition-on-icdar2015Baek et al.
Accuracy: 71.8
scene-text-recognition-on-svtBaek et al.
Accuracy: 87.5

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis | Papers | HyperAI