The AceMath RM Training Dataset was released by NVIDIA in 2024 for training Outcome Reward Models (ORMs), based on research detailed in "AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling" . It aims to enhance large models’ performance in mathematical reasoning tasks.

This dataset comprises 356,058 unique math problems, totaling 2,136,348 samples. Each problem corresponds to six distinct responses.
It primarily trains reward models (AceMath-7B and AceMath-72B) by evaluating response correctness to optimize mathematical reasoning capabilities.
In multiple benchmarks, models trained using this data demonstrated exceptional mathematical reasoning skills, surpassing various mainstream open-source and proprietary models.

Dataset Composition

The dataset mainly includes the following fields and structures:

• qid: Unique identifier for each question.
• message: A conversation list containing system prompts, user questions, and step-by-step solutions provided by assistants. Each entry specifies its role (role, e.g., system, user, assistant) and associated content (content).
• label: Binary classification indicating whether a response is correct/excellent (1 = Correct; 0 = Incorrect).

Data format consists of lists of JSON objects, directly loadable for model training. Every sample represents comparative data between one math problem and several candidate answers.

Citation

@article{acemath2024,
  title={AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling},
  author={Liu, Zihan and Chen, Yang and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei},
  journal={arXiv preprint},
  year={2024}
}