Command Palette
Search for a command to run...
Revisiting the Negative Data of Distantly Supervised Relation Extraction
Chenhao Xie Jiaqing Liang Jingping Liu Chengsong Huang Wenhao Huang Yanghua Xiao

Abstract
Distantly supervision automatically generates plenty of training samples for relation extraction. However, it also incurs two major problems: noisy labels and imbalanced training data. Previous works focus more on reducing wrongly labeled relations (false positives) while few explore the missing relations that are caused by incompleteness of knowledge base (false negatives). Furthermore, the quantity of negative labels overwhelmingly surpasses the positive ones in previous problem formulations. In this paper, we first provide a thorough analysis of the above challenges caused by negative data. Next, we formulate the problem of relation extraction into as a positive unlabeled learning task to alleviate false negative problem. Thirdly, we propose a pipeline approach, dubbed \textsc{ReRe}, that performs sentence-level relation detection then subject/object extraction to achieve sample-efficient training. Experimental results show that the proposed method consistently outperforms existing approaches and remains excellent performance even learned with a large quantity of false positive samples.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| relation-extraction-on-nyt10-hrl | HRL Takanobu et al. (2019) | F1: 64.4 |
| relation-extraction-on-nyt10-hrl | ReRe | F1: 73.95 |
| relation-extraction-on-nyt10-hrl | ReRe (exact) | F1: 73.4 |
| relation-extraction-on-nyt10-hrl | TPLinker Wang et al. (2020)* | F1: 72.45 |
| relation-extraction-on-nyt10-hrl | CasRel (exact) | F1: 70.11 |
| relation-extraction-on-nyt10-hrl | TPLinker Wang et al. (2020)*(exact) | F1: 71.93 |
| relation-extraction-on-nyt11-hrl | HRL | F1: 53.8 |
| relation-extraction-on-nyt11-hrl | ReRe (exact) | F1: 55.47 |
| relation-extraction-on-nyt11-hrl | RERE | F1: 56.23 |
| relation-extraction-on-nyt21 | CasRel (exact) | F1: 54.78 |
| relation-extraction-on-nyt21 | ReRe (exact) | F1: 58.88 |
| relation-extraction-on-nyt21 | TPLinker(exact) | F1: 57.33 |
| relation-extraction-on-nyt21 | ReRe | F1: 59.62 |
| relation-extraction-on-ske | CasRel (exact) | F1: 86.45 |
| relation-extraction-on-ske | ReRe (exact) | F1: 87.21 |
| relation-extraction-on-ske | TPLinker (exact) | F1: 84.32 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.