HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

Scalable Learning with Incremental Probabilistic PCA

{Adrian Barbu Boshi Wang}

Scalable Learning with Incremental Probabilistic PCA

Abstract

Incremental class learning is the classification problem of learning a model where instances from new object classes are added sequentially, and it is desired that the model be retrained only on the new classes with minimal training on the old classes. One major problem facing class incremental learning is catastrophic forgetting, where the updated model forgets the old classes and focuses only on the new classes.This paper proposes a simple and novel incremental class learning method that uses a self-supervised pretrained feature extractor to obtain meaningful features and trains Probabilistic PCA models on the extracted features for each class separately. The Mahalanobis distance is used to obtain the classification result, and an equivalent equation is derived to make the approach computationally affordable.Experiments on standard and large datasets show that the proposed approach outperforms existing state of the art incremental learning methods by a large margin.The fact that the model is trained on each class separately makes it applicable to training on very large datasets such as the whole ImageNet with more than 10,000 classes.

Benchmarks

BenchmarkMethodologyMetrics
class-incremental-learning-on-cifar-100-50-1PPCA-SWSL
Final Accuracy: 77.07
class-incremental-learning-on-cifar-100-50-1PPCA-CLIP
Final Accuracy: 69.71
class-incremental-learning-on-cifar-100-50-2PPCA-CLIP
Final Accuracy: 69.71
class-incremental-learning-on-cifar-100-50-2PPCA-SWSL
Final Accuracy: 77.07
incremental-learning-on-cifar-100-50-classes-3PPCA-SWSL
Final Accuracy: 77.07
incremental-learning-on-cifar-100-50-classes-3PPCA-CLIP
Final Accuracy: 69.71
incremental-learning-on-imagenet-10k-5225PPCA-CLIP
Final Accuracy: 35.42
incremental-learning-on-imagenet-500-classes-1PPCA-CLIP
Final Accuracy: 71.25
incremental-learning-on-imagenet-500-classes-2PPCA-CLIP
Final Accuracy: 71.25

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
Scalable Learning with Incremental Probabilistic PCA | Papers | HyperAI