HyperAIHyperAI

Command Palette

Search for a command to run...

HumanEval

Date

HumanEval (Code) is a code generation benchmark proposed by OpenAI, comprising 164 hand-written programming problems and their corresponding unit tests. The HumanEval score refers to the proportion of functions generated by the model that pass all test cases, making it one of the most commonly used benchmark metrics in the field of code generation. It evaluates functional correctness, rather than code style or efficiency. It is typically reported as a percentage or a score between 0 and 1, where higher values are better. When using it, one must specify whether the metric is pass@1 or pass@k, along with settings such as sampling temperature, to ensure fair comparison.

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing

HyperAI Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp