HyperAI
Command Palette
Search for a command to run...
Cross-modal Generation
Generating content in one modality conditioned on another, such as image captioning, text-to-image, or visual dialogue.
Leaderboard
100 models total
Benchmarks
COCO
ExpansionNet v2
MIMIC-CXR
VizWiz 2020 test-dev
Flickr-30K
No-Caps
COCO-Captions
OSCAR
nocaps out-of-domain
PaLI
nocaps in-domain
VinVL (Microsoft Cognitive Services + MSR)
nocaps entire
nocaps near-domain
GIT2, Single Model
VIST
MSR-VTT
mPLUG-2
AudioCaps
VAST
MSVD
MaMMUT
YouCook2
Univl
Clotho
ZerAuCap
MS-COCO
NeuSyRE
IU X-Ray
TextCAPS
VisDial v0.9 val
9xFGA (VGG)
HumanML3D
CHEXPERT-PLUS
CLEVR-Change
CUB
VQ-Diffusion-F
VizWiz 2020 test
Gibson
HM3D
nocaps-XD entire
GIT
RefCOCO-g
Spot-the-Diff
ActivityNet-Captions
nocaps-val-in-domain
nocaps-val-overall
nocaps-XD in-domain
GIT
nocaps-XD near-domain
GIT
nocaps-XD out-of-domain
GIT
TextCaps 2020
VATEX
VALOR
BoFiT
Multi-Modal-CelebA-HQ
nocaps-val-near-domain
nocaps-val-out-domain
XM3600
Image Paragraph Captioning
SCICAP
CNN+LSTM (Vision only, First sentence)
CMD-AD
COCO Karpathy
COCO2017 caption
CoMMuTE
Multi-30k
Nymeria
OB-Radix
TV-AD
VggCaps
ChestXray8
Conceptual Captions
ClipCap (MLP + GPT2 tuning)
Daily-PP
DOCCI
Flickr
Flickr30k Captions test
Unified VLP
MS-CXR
Nr3D
Oxford 102 Flowers
RAT-Diffusion
RefCOCO
ScanRefer
VinDr-CXR
WIKIWEB2M
Chart-to-Text
ChartMind
ChEBI-20
GIT-Mol
CLEVR-DC
DiagramQG
NBA-Identity
OpenCQA
VC-NBA-2022
Visual Genome
ControlCap
WHOOPS
BDD-X
COCO-2017
Ego4D
EGTEA+ Gaze
Flickr-8k
Image-Chat_AVD
Image-Editing-Request
MSRVTT-CTN
MSVD-CTN
Photo-Chat_AVD
SN-Long
VisualMRC
BrainHub
HIVAU-70K
Image-Edit-Request
L+M-24
MAD-Eval
ScanQA
VENUS
AVA-Captions
Birds-to-Words
Clotho v1
Clotho-v2
COCO
Parti Finetuned
COCO-Cap
COCO-EE
COMPOSITION-CAP
EchoVLM dataset
Flickr30k → MSCOCO
HowToNarrate
iGround
L+M
LingoQA
LLaVAR-2-Cap
LLaVAR-2-VQA
MS COCO (Karpathy Split)
MSCOCO → Flickr30k
MUSIC-QA
nocaps val
Prismer
PHOENIX-2014T
R2R
REVERIE
Stanford Image Paragraph dataset
ViTT
Vid2Seq (VidChapters-7M PT)
A-OKVQA
ActivityNet-Caption
AVSD-DSTC10
AVSD-DSTC7
AVSD-DSTC8
ChartCap
COCO test
COCO-Caption
COCO Captions test
From Captions to Visual Concepts and Back
COCO-LN
CSL-Daily
DRAMA
E.E.
Flickr30k Entities
Flickr30k-LN
IC
Image Caption
Image-Chat
Instruct-V2Xum
K-VQG
KnowCap Test
Kollenda et al.
LAION-COCO
Parti Finetuned
LEVIR-CC
LLaVA-Med
Localized Narratives
Longitudinal-MIMIC
Med-Trinity
Motion-X
MSCOCO Test
Music4way-Any2T
Music4way-MI2T
Music4way-MusicCaps
Music4way-MV2T
MusicCaps
Narratives
Natural Scenes Dataset (NSD)
OK-VQA
OpenImages-I
Reasoning
RefCOCO+
ShareGPT4V, DCI
Shot2Story20K
SoccerNet-v2
Spot-the-Change
Spotlight
TVC
UCCD
UCM-Captions
UniMER-Test
VEGA
VG
VideoXum
VisDial v1.0 test-std
5xFGA + LS*+
VQG-COCO
YOUCOOKII
ACD
AIC-ICC
AppealImage
BanglaLekhaImageCaptions
CNN + 1D CNN
BCT-CHR
BenSMOT
BlendedSkillTalk
BLiSS
Boxing (BX)
CB-300K
CheXpert
ChinaOpen-1k
GVT
CLD
ClothoCaps
CMER-Bench
Colors
BiLSTMS on color generation
Conceptual
ConvAI2
CoVLA
CRG
CTRG-Brain
CUB-200
CUVA
DnD Group Gesture dataset
DNICC19k
DocParsing
Dubai-CC
DVQA
EE4D-DescCoach
Ego-Exo4D
Empathetic Dialogues
FetusR
Figure Skating (FS)
Flickr30k Test
Flickr30k-lite
FlickrStyle10K
CapDec
FSVQA
GigaHands
GoodNews
GrowthFetus
HC-STVG
He et al.
Hindi Visual Genome (Challenge Set)
Hindi Visual Genome (Test Set)
HouseTour dataset
How2Sign
IER
J-MID
Kinetics-GEB+
ActBERT-revised
KIT-ML
LeBel's dataset
LN COCO
M3-20M
MAD dataset
MAviS-Bench
Meme-Image-No-Text
MEMECAP
MM-AVS
MMTT
Mol-Instructions
MORE
MS-COCO Karpathy test split
MSVD-Indonesian
Multi-Modal CelebA-HQ
NSD
NuInteract
nuScenes
NWPU-Captions
NYTimes800k
OmniDiff
OmniDrive-nuScenes
PAd1M
Paragraph Caption
PATENTDESC-355K
PCDes
Peir Gross
BiomedGPT
PETARSeg-11k
Previous Caption
proprietary longitudinal chest CT dataset
PV-VTT
QEVD-bio-fit-coach
QEVD-fit-coach
Real-world audio-visual video evaluation set
Recipe1M
Ref-L4
RQA-70K
RSICD
RxR
ScanNet++
ScanQAval
SceneCap
Semantic-HOI
ShareGPT4V & DCI
SoccerNet-Caption
Spot-the-Dif
STD
SummScreen3D
SurgVLM
Sydney-Captions
Syned
Syned + EE
TIB
Trailer-Test
UCFCAP
UMD
UniKnowCap Test
V2C
VAR test
VDC
VidChapters-7M
VietPET-RoI
VIEWS
VISTA
Visual News
VQA-E
VRG
W3DA
Wizard of Wikipedia
WOMD-Reasoning
XMSMO-News
Yo'LLaVA
YouCookII test