HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling

{eep S Attree}

Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling

Abstract

This paper presents a strong set of results for resolving gendered ambiguous pronouns on the Gendered Ambiguous Pronouns shared task. The model presented here draws upon the strengths of state-of-the-art language and coreference resolution models, and introduces a novel evidence-based deep learning architecture. Injecting evidence from the coreference models compliments the base architecture, and analysis shows that the model is not hindered by their weaknesses, specifically gender bias. The modularity and simplicity of the architecture make it very easy to extend for further improvement and applicable to other NLP problems. Evaluation on GAP test data results in a state-of-the-art performance at 92.5{%} F1 (gender bias of 0.97), edging closer to the human performance of 96.6{%}. The end-to-end solution presented here placed 1st in the Kaggle competition, winning by a significant lead.

Benchmarks

BenchmarkMethodologyMetrics
coreference-resolution-on-gap-1ProBERT
Bias (F/M): 0.97
Feminine F1 (F): 91.1
Masculine F1 (M): 94.0
Overall F1: 92.5

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling | Papers | HyperAI