Command Palette
Search for a command to run...
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering
Ankit Pal Logesh Kumar Umapathi Malaikannan Sankarasubbu
Abstract
This paper introduces MedMCQA, a new large-scale, Multiple-Choice Question Answering (MCQA) dataset designed to address real-world medical entrance exam questions. More than 194k high-quality AIIMS \& NEET PG entrance exam MCQs covering 2.4k healthcare topics and 21 medical subjects are collected with an average token length of 12.77 and high topical diversity. Each sample contains a question, correct answer(s), and other options which requires a deeper language understanding as it tests the 10+ reasoning abilities of a model across a wide range of medical subjects \& topics. A detailed explanation of the solution, along with the above information, is provided in this study.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| multiple-choice-question-answering-mcqa-on-21 | SciBERT (Beltagy et al., 2019) | Dev Set (Acc-%): 0.39 Test Set (Acc-%): 0.39 |
| multiple-choice-question-answering-mcqa-on-21 | BioBERT (Lee et al.,2020) | Dev Set (Acc-%): 0.38 Test Set (Acc-%): 0.37 |
| multiple-choice-question-answering-mcqa-on-21 | BERT (Devlin et al., 2019)-Base | Dev Set (Acc-%): 0.35 Test Set (Acc-%): 0.33 |
| multiple-choice-question-answering-mcqa-on-21 | PubmedBERT(Gu et al., 2022) | Dev Set (Acc-%): 0.40 Test Set (Acc-%): 0.41 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.