HyperAIHyperAI

Command Palette

Search for a command to run...

4 months ago

"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection

William Yang Wang

"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection

Abstract

Automatic fake news detection is a challenging problem in deception detection, and it has tremendous real-world political and social impacts. However, statistical approaches to combating fake news has been dramatically limited by the lack of labeled benchmark datasets. In this paper, we present liar: a new, publicly available dataset for fake news detection. We collected a decade-long, 12.8K manually labeled short statements in various contexts from PolitiFact.com, which provides detailed analysis report and links to source documents for each case. This dataset can be used for fact-checking research as well. Notably, this new dataset is an order of magnitude larger than previously largest public fake news datasets of similar type. Empirically, we investigate automatic fake news detection based on surface-level linguistic patterns. We have designed a novel, hybrid convolutional neural network to integrate meta-data with text. We show that this hybrid approach can improve a text-only deep learning model.

Benchmarks

BenchmarkMethodologyMetrics
fake-news-detection-on-liarCNNs
Test Accuracy: 0.27
Validation Accuracy: 0.26
fake-news-detection-on-liarHybrid CNNs (Text + Speaker)
Test Accuracy: 0.248
Validation Accuracy: 0.277
fake-news-detection-on-liarBi-LSTMs
Test Accuracy: 0.233
Validation Accuracy: 0.223
fake-news-detection-on-liarHybrid CNNs (Text + All)
Test Accuracy: 0.274
Validation Accuracy: 0.247

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection | Papers | HyperAI