MMPR (Multimodal Preference Dataset) 是由上海人工智能实验室、复旦大学、南京大学、香港中文大学、清华大学和商汤科技的研究团队于 2024 年共同发布的一个大规模的多模态偏好数据集，相关论文成果为「Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization」。该数据集包含 75 万无明确正确答案样本和 250 万有明确正确答案样本。样本覆盖了多个领域，如 VQA 、科学、图表、数学、 OCR 和文档，以确保多样性。在构建数据集时，研究者特别注意避免因启发式规则的局限性而导致的误报负响应，特别是在通用 VQA 和文档领域。数据集的设计旨在提高模型在多模态推理任务中的表现，同时避免训练过程中的潜在负面影响。

Citation

If you find this project useful in your research, please consider citing: “`BibTeX @article{wang2024mpo, title={Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization}, author={Wang, Weiyun and Chen, Zhe and Wang, Wenhai and Cao, Yue and Liu, Yangzhou and Gao, Zhangwei and Zhu, Jinguo and Zhu, Xizhou and Lu, Lewei and Qiao, Yu and Dai, Jifeng}, journal={arXiv preprint arXiv:2411.10442}, year={2024} } @article{chen2023internvl, title={InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks}, author={Chen, Zhe and Wu, Jiannan and Wang, Wenhai and Su, Weijie and Chen, Guo and Xing, Sen and Zhong, Muyan and Zhang, Qinglong and Zhu, Xizhou and Lu, Lewei and Li, Bin and Luo, Ping and Lu, Tong and Qiao, Yu and Dai, Jifeng}, journal={arXiv preprint arXiv:2312.14238}, year={2023} } @article{chen2024far, title={How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites}, author={Chen, Zhe and Wang, Weiyun and Tian, Hao and Ye, Shenglong and Gao, Zhangwei and Cui, Erfei and Tong, Wenwen and Hu, Kongzhi and Luo, Jiapeng and Ma, Zheng and others}, journal={arXiv preprint arXiv:2404.16821}, year={2024} }