摘要
对比语言-图像预训练（Contrastive Language-Image Pre-training，简称CLIP）因其在多种场景中的潜力而受到越来越多的关注。本文提出了一种名为EVA-CLIP的模型系列，显著提高了CLIP训练的效率和效果。我们的方法引入了新的表示学习、优化和增强技术，使得EVA-CLIP在参数数量相同的情况下，相比之前的CLIP模型实现了更优的性能，同时大幅降低了训练成本。特别值得一提的是，我们最大的50亿参数模型EVA-02-CLIP-E/14+仅通过90亿样本就达到了ImageNet-1K验证集上的82.0%零样本顶级准确率。较小的EVA-02-CLIP-L/14+模型拥有4.3亿参数和60亿样本，也在ImageNet-1K验证集上达到了80.4%的零样本顶级准确率。为了促进开放访问和开放研究，我们将完整的EVA-CLIP套件发布到社区，地址为https://github.com/baaivision/EVA/tree/master/EVA-CLIP。
代码仓库
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| image-classification-on-objectnet | EVA-02-CLIP-E/14+ | Top-1 Accuracy: 79.6 |
| zero-shot-action-recognition-on-ucf101 | EVA-CLIP-E/14+ | Top-1 Accuracy: 83.1 |
| zero-shot-transfer-image-classification-on-1 | EVA-CLIP-E/14+ | Accuracy (Private): 82 |
| zero-shot-transfer-image-classification-on-17 | EVA-CLIP-E/14+ | Top 1 Accuracy: 94.9 |
| zero-shot-transfer-image-classification-on-3 | EVA-CLIP-E/14+ | Accuracy (Private): 75.7 |
| zero-shot-transfer-image-classification-on-4 | EVA-CLIP-E/14+ | Accuracy: 94.5 |
| zero-shot-transfer-image-classification-on-5 | EVA-CLIP-E/14+ | Accuracy (Private): 82.1 |
| zero-shot-transfer-image-classification-on-6 | EVA-CLIP-E/14+ | Accuracy (Private): 79.6 |
| zero-shot-transfer-image-classification-on-8 | EVA-CLIP-E/14+ | Accuracy (Private): 71.6 |