HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering

{Feng Jiang Jian-Guang Lou Chin-Yew Lin Zhiwei Yu Qian Liu Shuang Chen}

ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering

Abstract

We present Retriever-Transducer-Checker (ReTraCk), a neural semantic parsing framework for large scale knowledge base question answering (KBQA). ReTraCk is designed as a modular framework to maintain high flexibility. It includes a retriever to retrieve relevant KB items efficiently, a transducer to generate logical form with syntax correctness guarantees and a checker to improve transduction procedure. ReTraCk is ranked at top1 overall performance on the GrailQA leaderboard and obtains highly competitive performance on the typical WebQuestionsSP benchmark. Our system can interact with users timely, demonstrating the efficiency of the proposed framework.

Benchmarks

BenchmarkMethodologyMetrics
knowledge-base-question-answering-on-1ReTraCk Oracle EL
F1: 74.7
Hits@1: 74.6
knowledge-base-question-answering-on-1ReTraCk
F1: 71
Hits@1: 71.6
knowledge-base-question-answering-on-grailqaReTraCk
Compositional EM: 61.5
Compositional F1: 70.9
I.I.D. EM: 84.4
I.I.D. F1: 87.5
Overall EM: 58.1
Overall F1: 65.3
Zero-shot EM: 44.6
Zero-shot F1: 52.5

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp