Dataset Introduction

AceMath-RewardBench is a benchmark dataset for evaluating the capabilities of mathematical reward models. This benchmark employs a best-of-N (N=8) setting and covers the following seven mathematical inference datasets: GSM8K, Math500, Minerva Math, Gaokao 2023 En, OlympiadBench, College Math, and MMLU STEM. The datasets are designed to measure the ability of reward models to select the correct solution from multiple candidate solutions. Each example contains a math problem and 64 attempt solutions from 8 different language models, varying in quality. The dataset provides a true score for each solution, along with additional metadata such as problem difficulty and subject area. The evaluation benchmarks focus on two criteria: diversity (64 answers per problem from 8 different models) and robustness (evaluated by averaging results from 100 random seeds).

Dataset composition

The dataset contains 7 subsets, each corresponding to a different mathematical reasoning task. The specific number of questions is as follows: – GSM8K: 1319 Questions – Math500: 500 Questions – Minerva Math: 272 Questions – Gaokao 2023 En: 385 questions OlympiadBench: 675 questions – College Math: 2818 Questions – MMLU STEM: 3018 questions Each example's data format includes the following fields:question(Text of mathematical problems) code(Complete list of model solutions) gt(Standard Answer) pred(A list of predicted answers extracted from each solution) score(A list of Boolean values indicating whether each solution matches the standard answer) idx(index), report(Report), gt_cot(The thought process behind the standard answer). The dataset features diversity (64 answers from 8 different models) and robustness (8 candidates are randomly selected from 64 candidates using 100 random seeds, and the average result is reported). This dataset is suitable for evaluating the performance of mathematical reward models, particularly in the optimal N-to-1 setting. License information: This dataset is licensed under the Creative Commons Attribution Non-Commercial 4.0 International (CC-BY-NC-4.0) license and is intended for non-commercial use only.

Citation

@article{acemath2024, title={AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling}, author={Liu, Zihan and Chen, Yang and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei}, journal={arXiv preprint}, year={2024} }