Language Modelling On The Pile

评估指标

Bits per byte

评测结果

各个模型在此基准测试上的表现结果

		Paper Title
GPT-2 Small 124M (pre-trained)	1.2253	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
GPT-2 Medium 355M (pre-trained)	1.0928	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
GPT-2 Large 774M (pre-trained)	1.0828	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
GPT-2 XL 1.5B (pre-trained)	1.0468	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
GPT-3 Ada 350M (pre-trained)	0.9631	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
GPT-3 Babbage 1.3B (pre-trained)	0.8718	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Test-Time Fine-Tuning with SIFT + GPT-2 (124M)	0.862	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
GPT-2 Large 774M (test-time training on nearest neighbors)	0.85	Test-Time Training on Nearest Neighbors for Large Language Models
Llama-3.2-Instruct 1B	0.807	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
GPT-3 Curie 6.7B (pre-trained)	0.7980	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Test-Time Fine-Tuning with SIFT + GPT-2 (774M)	0.762	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
GPT-3	0.742	GLM-130B: An Open Bilingual Pre-trained Model
Llama-3.2-Instruct 3B	0.737	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
Gemma-2 2B	0.721	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
GPT-3 Davinci 175B (pre-trained)	0.7177	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Llama-3.2 1B	0.697	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
Phi-3 3.8B	0.679	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
Phi-3 7B	0.678	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
Gemma-2 9B	0.670	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
Phi-3 14B	0.651	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs

0 of 39 row(s) selected.