MathVista is a comprehensive mathematical reasoning benchmark in a visual environment. It consists of three newly created datasets, IQTest, FunctionQA, and PaperQA, which address missing visual domains and are designed to evaluate logical reasoning on jigsaw test images, algebraic reasoning on function images, and scientific reasoning on academic paper images, respectively. In total, MathVista contains 6,141 examples collected from 31 different datasets.

Citation

@inproceedings{lu2024mathvista,
author = {Lu, Pan and Bansal, Hritik and Xia, Tony and Liu, Jiacheng and Li, Chunyuan and Hajishirzi, Hannaneh and Cheng, Hao and Chang, Kai-Wei and Galley, Michel and Gao, Jianfeng},
title = {MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts},
booktitle = {International Conference on Learning Representations (ICLR)},
year = {2024}
}