HelpSteer3 是由英伟达于 2025 年发布的一个人类偏好数据集，相关论文成果为「HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages」，旨在通过人类反馈和强化学习技术提升模型对用户提示的响应能力。 该数据集包含 40,476 个偏好样本，每个样本包含一个领域、语言、上下文、两个回复、以及 2 个回复之间的总体偏好评分和最多 3 位标注者的个人偏好评分，其包含了多语言数据（中文、韩文、法语、西班牙文、日本语、德语、俄语、葡萄牙语、意大利语、越南语、荷兰语）。

Citation

@misc{wang2025helpsteer3preferenceopenhumanannotatedpreference,
title={HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages},
author={Zhilin Wang and Jiaqi Zeng and Olivier Delalleau and Hoo-Chang Shin and Felipe Soares and Alexander Bukharin and Ellie Evans and Yi Dong and Oleksii Kuchaiev},
year={2025},
eprint={2505.11475},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.11475},
}
@misc{wang2025helpsteer3humanannotatedfeedbackedit,
title={HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks},
author={Zhilin Wang and Jiaqi Zeng and Olivier Delalleau and Daniel Egert and Ellie Evans and Hoo-Chang Shin and Felipe Soares and Yi Dong and Oleksii Kuchaiev},
year={2025},
eprint={2503.04378},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2503.04378},
}
@misc{wang2025rlbffbinaryflexiblefeedback,
title={RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards},
author={Zhilin Wang and Jiaqi Zeng and Olivier Delalleau and Ellie Evans and Daniel Egert and Hoo-Chang Shin and Felipe Soares and Yi Dong and Oleksii Kuchaiev},
year={2025},
eprint={2509.21319},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2509.21319},
}