摘要
在基础模型出现之后，针对长尾学习任务的微调范式引起了广泛关注。然而，微调对长尾学习性能的影响尚未得到明确量化。本文揭示了，重度微调可能会导致尾部类别的性能显著下降，而轻度微调则更为有效。原因归结为重度微调引起的类别条件不一致。基于上述观察，我们开发了一种低复杂度且准确的长尾学习算法LIFT（Low-Complexity and Accurate Long-Tail Learning with Adaptive Lightweight Fine-Tuning），旨在通过自适应轻度微调实现快速预测和紧凑模型。实验结果清楚地验证了，与现有最先进方法相比，LIFT不仅显著减少了训练时间和学习参数的数量，还提高了预测准确性。该算法的实现代码可在https://github.com/shijxcs/LIFT 获取。
代码仓库
shijxcs/lift
官方
pytorch
GitHub 中提及
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| long-tail-learning-on-cifar-100-lt-r-10 | LIFT (ViT-B/16, ImageNet-21K pre-training) | Error Rate: 8.7 |
| long-tail-learning-on-cifar-100-lt-r-10 | LIFT (ViT-B/16, CLIP) | Error Rate: 15.1 |
| long-tail-learning-on-cifar-100-lt-r-100 | LIFT (ViT-B/16, ImageNet-21K pre-training) | Error Rate: 10.9 |
| long-tail-learning-on-cifar-100-lt-r-100 | LIFT (ViT-B/16, CLIP) | Error Rate: 18.3 |
| long-tail-learning-on-cifar-100-lt-r-50 | LIFT (ViT-B/16, CLIP) | Error Rate: 16.9 |
| long-tail-learning-on-cifar-100-lt-r-50 | LIFT (ViT-B/16, ImageNet-21K pre-training) | Error Rate: 9.8 |
| long-tail-learning-on-imagenet-lt | LIFT (ViT-B/16) | Top-1 Accuracy: 78.3 |
| long-tail-learning-on-imagenet-lt | LIFT (ViT-L/14) | Top-1 Accuracy: 82.9 |
| long-tail-learning-on-inaturalist-2018 | LIFT (ViT-B/16) | Top-1 Accuracy: 80.4% |
| long-tail-learning-on-inaturalist-2018 | LIFT (ViT-L/14) | Top-1 Accuracy: 85.2% |
| long-tail-learning-on-inaturalist-2018 | LIFT (ViT-L/14@336px) | Top-1 Accuracy: 87.4% |
| long-tail-learning-on-places-lt | LIFT (ViT-L/14) | Top-1 Accuracy: 53.7 |
| long-tail-learning-on-places-lt | LIFT (ViT-B/16) | Top-1 Accuracy: 52.2 |