HyperAIHyperAI

Command Palette

Search for a command to run...

5 months ago

Neural RST-based Evaluation of Discourse Coherence

Grigorii Guz; Peyman Bateni; Darius Muglich; Giuseppe Carenini

Neural RST-based Evaluation of Discourse Coherence

Abstract

This paper evaluates the utility of Rhetorical Structure Theory (RST) trees and relations in discourse coherence evaluation. We show that incorporating silver-standard RST features can increase accuracy when classifying coherence. We demonstrate this through our tree-recursive neural model, namely RST-Recursive, which takes advantage of the text's RST features produced by a state of the art RST parser. We evaluate our approach on the Grammarly Corpus for Discourse Coherence (GCDC) and show that when ensembled with the current state of the art, we can achieve the new state of the art accuracy on this benchmark. Furthermore, when deployed alone, RST-Recursive achieves competitive accuracy while having 62% fewer parameters.

Code Repositories

grig-guz/coherence-rst
Official
pytorch
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
coherence-evaluation-on-gcdc-rst-accuracyRST-Ensemble
Accuracy: 55.39
coherence-evaluation-on-gcdc-rst-accuracyRST-Recursive
Accuracy: 53.04
coherence-evaluation-on-gcdc-rst-f1RST-Ensemble
Average F1: 46.98
coherence-evaluation-on-gcdc-rst-f1RST-Recursive
Average F1: 44.30

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
Neural RST-based Evaluation of Discourse Coherence | Papers | HyperAI