该数据集由浙江大学、中科院软件研究所、上海科技大学等机构于 2024 年联合推出，相关论文成果为「Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model」。 数据集共包含 11,193 个带有相关问题的抽象图像，涵盖了仪表板、路线图、图表、表格、流程图、关系图、视觉谜题和 2D 平面图等 8 大类别，此外还有额外的 62,476 条数据用于微调模型。

Citation

@inproceedings{zhang-etal-2024-multimodal,
title = "Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model",
author = "Zhang, Wenqi  and
Cheng, Zhenglin  and
He, Yuanyu  and
Wang, Mengna  and
Shen, Yongliang  and
Tan, Zeqi  and
Hou, Guiyang  and
He, Mingqian  and
Ma, Yanna  and
Lu, Weiming  and
Zhuang, Yueting",
booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
year = "2024",
address = "Miami, Florida, USA",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.emnlp-main.1072/",
pages = "19228--19252"}