Command Palette
Search for a command to run...
Sergei Bogdanov; Alexandre Constantin; Timothée Bernard; Benoit Crabbé; Etienne Bernard
Abstract
Large Language Models (LLMs) have shown impressive abilities in data annotation, opening the way for new approaches to solve classic NLP problems. In this paper, we show how to use LLMs to create NuNER, a compact language representation model specialized in the Named Entity Recognition (NER) task. NuNER can be fine-tuned to solve downstream NER problems in a data-efficient way, outperforming similar-sized foundation models in the few-shot regime and competing with much larger LLMs. We find that the size and entity-type diversity of the pre-training dataset are key to achieving good performance. We view NuNER as a member of the broader family of task-specific foundation models, recently unlocked by LLMs.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| few-shot-ner-on-few-nerd-inter | NuNER | 10 way 1~2 shot: 66.54±0.40 10 way 5~10 shot: 71.04±0.14 5 way 1~2 shot: 67.37±0.31 5 way 5~10 shot: 73.50±0.09 |
| few-shot-ner-on-few-nerd-intra | NuNER | 10 way 1~2 shot: 57.63±0.38 10 way 5~10 shot: 62.99±0.27 5 way 1~2 shot: 62.48±0.28 5 way 5~10 shot: 69.16±0.28 |
| named-entity-recognition-ner-on-ncbi-disease | NuNER Zero Span | F1: 61.1 |
| named-entity-recognition-ner-on-ontonotes-v5 | NuNER | F1: 89.1 Precision: 87.8 Recall: 90.5 |
| named-entity-recognition-on-few-nerd-sup | NuNER | F1-Measure: 69.4 Precision: 67.8 Recall: 71.1 |
| zero-shot-named-entity-recognition-ner-on | NuNerZero Span | Entity F1: 60.2 |
| zero-shot-named-entity-recognition-ner-on-1 | NuNERZero span | AI: 61.7 Literature: 64.9 Music: 69.9 Politics: 71.7 Science: 65.4 |
| zero-shot-named-entity-recognition-ner-on-2 | NuNER Zero Span | Entity F1: 24.9 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.