Home Console Docs News Papers Tutorials Datasets Wiki SOTA LLM Models GPU Leaderboard Events

English

Code Generation On Conala

Metrics

BLEU

Exact Match Accuracy

Results

Performance results of various models on this benchmark

			Paper Title	Repository
PanGu-Coder-FT-I	44.32	-	Fine-Tuning Large Language Models for Answering Programming Questions with Code Snippets	-
RoBERTaMarian	35.74	13.8	Leveraging pre-trained language models for code generation	-
MarianCG	34.43	10.2	MarianCG: a code generation transformer model inspired by machine translation	-
TranX + BERT w/mined	34.2	5.8	The impact of lexical and grammatical processing on generating code from natural language
BERT + TAE	33.41	-	Code Generation from Natural Language with Less Prior and More Monolingual Data
BERTMarian	32.46	12.40	Leveraging pre-trained language models for code generation	-
External Knowledge With API + Reranking	32.26	-	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation
External Knowledge With API	30.69	-	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation
BART W/ Mined	30.55	-	Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation
ELECTRAMarian	30.18	10.0	Leveraging pre-trained language models for code generation	-
Reranker	30.11	2.8	Reranking for Neural Semantic Parsing	-
LUKEMarian	29.83	7.6	Leveraging pre-trained language models for code generation	-
BART Base	26.24	-	Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation
TranX	24.30	-	TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation

0 of 14 row(s) selected.

Code Generation On Conala | SOTA | HyperAI