HyperAIHyperAI
HomeConsoleDocsNewsPapersTutorialsDatasetsWikiSOTALLM ModelsGPU LeaderboardEvents
Search
About
HyperAIHyperAI

Command Palette

Search for a command to run...

Logical Reasoning On Lingoly

Metrics

Delta_NoContext
Exact Match Accuracy

Results

Performance results of various models on this benchmark

Paper TitleRepository
Claude Opus28.8%46.3%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-4o25.1%37.6%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Gemini 1.5 Pro23.4%32.1%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-421.5%33.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Command R+11.6%21.5%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-3.511.2%21.2%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Mixtral 8x7B6.4%14.2%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 8B4.9%11.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 70B2.9%10.3%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Gemma 7B2.2%4.9%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 2 70B1.1%6.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
0 of 11 row(s) selected.
Logical Reasoning On Lingoly | SOTA | HyperAI