Text To Sql On Spider

评估指标

Exact Match Accuracy (Dev)

Exact Match Accuracy (Test)

Execution Accuracy (Test)

评测结果

各个模型在此基准测试上的表现结果

				Paper Title	Repository
datagpt-sql-7B + InvalidSQL-Feedback	81.6	-	-	DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL	-
RESDSQL-3B + NatSQL	80.5	72.0	79.9	RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL
datagpt-sql-7B	80.3	-	-	DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL	-
T5-SR	77.2	72.4	75.2	T5-SR: A Unified Seq-to-Seq Decoding Strategy for Semantic Parsing
Graphix-3B+PICARD	77.1	-	77.6	Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing
T5-3B + PICARD	75.5	71.9	75.1	PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models
DAIL-SQL + GPT-4 + Self-Consistency	74.4	-	86.6	Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation
SADGA + GAP	73.1	70.1	-	SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL
RATSQL + GAP	71.8	-	-	Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training
T5-3B+NatSQL+Token Preprocessing	69.4	-	78	Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques
RYANSQL (BERT)	66.6	58.2	-	RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases
MAPO + TABERTLarge (K = 3)	64.5	-	-	TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
MSc-SQL	-	-	84.7	MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation
code-davinci-002 175B (LEVER)	-	-	-	LEVER: Learning to Verify Language-to-Code Generation with Execution
PET-SQL	-	66.6	87.6	PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency
MARLO + Claude 2.1	-	-	84.0	Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection	-
DIN-SQL + GPT-4	-	60	85.3	DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction
C3 + ChatGPT + Zero-Shot	-	-	82.3	C3: Zero-shot Text-to-SQL with ChatGPT
XiYan-SQL	-	-	89.65	A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL
DELLM + GPT-4	-	-	-	Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM

0 of 20 row(s) selected.