HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

FinQA: A Dataset of Numerical Reasoning over Financial Data

FinQA: A Dataset of Numerical Reasoning over Financial Data

Abstract

The sheer volume of financial statements makes it difficult for humans to access and analyze a business's financials. Robust numerical reasoning likewise faces unique challenges in this domain. In this work, we focus on answering deep questions over financial data, aiming to automate the analysis of a large corpus of financial documents. In contrast to existing tasks on general domain, the finance domain includes complex numerical reasoning and understanding of heterogeneous representations. To facilitate analytical progress, we propose a new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts. We also annotate the gold reasoning programs to ensure full explainability. We further introduce baselines and conduct comprehensive experiments in our dataset. The results demonstrate that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge and in complex multi-step numerical reasoning on that knowledge. Our dataset -- the first of its kind -- should therefore enable significant, new community research into complex application domains. The dataset and code are publicly available\url{https://github.com/czyssrs/FinQA}.

Code Repositories

czyssrs/finqa
Official
pytorch
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
question-answering-on-finqaFinQANet (BERT-large)
Execution Accuracy: 57.43
Program Accuracy: 55.52
question-answering-on-finqaFinQANet (RoBERTa-large)
Execution Accuracy: 65.05
Program Accuracy: 63.52
question-answering-on-finqaFinQANet (FinBert )
Execution Accuracy: 53.71
Program Accuracy: 51.71

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
FinQA: A Dataset of Numerical Reasoning over Financial Data | Papers | HyperAI