MultiBench is a systematic and unified large-scale multimodal learning benchmark covering 15 datasets, 10 modalities, 20 prediction tasks, and 6 research areas. MultiBench provides an automated end-to-end machine learning pipeline that simplifies and standardizes data loading, experimental setup, and model evaluation. To achieve holistic evaluation, MultiBench provides evaluation methods to study (1) generalization; (2) time and space complexity, and (3) modality robustness.

Citation

If you find our work useful for your research, please consider citing it.