Reading Comprehension On Race

评估指标

Accuracy

Accuracy (High)

Accuracy (Middle)

评测结果

各个模型在此基准测试上的表现结果

				Paper Title
ALBERTxxlarge+DUMA(ensemble)	89.8	92.6	88.7	DUMA: Reading Comprehension with Transposition Thinking
Megatron-BERT (ensemble)	90.9	90.0	93.1	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Megatron-BERT	89.5	88.6	91.8	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
B10-10-10	85.7	84.4	88.8	Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
XLNet	-	84.0	88.6	XLNet: Generalized Autoregressive Pretraining for Language Understanding
RoBERTa	83.2	81.3	86.5	RoBERTa: A Robustly Optimized BERT Pretraining Approach
LLaMA 65B (zero-shot)	-	51.6	67.9	LLaMA: Open and Efficient Foundation Language Models
PaLM 540B (zero-shot)	-	49.1	68.1	PaLM: Scaling Language Modeling with Pathways
LLaMA 33B (zero-shot)	-	48.3	64.1	LLaMA: Open and Efficient Foundation Language Models
PaLM 62B (zero-shot)	-	47.5	64.3	PaLM: Scaling Language Modeling with Pathways
LLaMA 13B (zero-shot)	-	47.2	61.6	LLaMA: Open and Efficient Foundation Language Models
LLaMA 7B (zero-shot)	-	46.9	61.1	LLaMA: Open and Efficient Foundation Language Models
GPT-3 175B (zero-shot)	-	45.5	-	Language Models are Few-Shot Learners
PaLM 8B (zero-shot)	-	42.3	57.9	PaLM: Scaling Language Modeling with Pathways
Bloomberg GPT (one-shot)	-	41.74	54.32	BloombergGPT: A Large Language Model for Finance
BLOOM 176B (one-shot)	-	39.14	52.3	BloombergGPT: A Large Language Model for Finance
OPT 66B (one-shot)	-	37.02	47.42	BloombergGPT: A Large Language Model for Finance
GPT-NeoX (one-shot)	-	34.33	41.23	BloombergGPT: A Large Language Model for Finance
DeBERTalarge	86.8	-	-	DeBERTa: Decoding-enhanced BERT with Disentangled Attention
GPT-3 175B (0-shot)	-	-	58.4	Language Models are Few-Shot Learners

0 of 24 row(s) selected.