Command Palette
Search for a command to run...
Jaewoo Park Yoon Gyo Jung Andrew Beng Jin Teoh

Abstract
Detecting out-of-distribution (OOD) samples are crucial for machine learning models deployed in open-world environments. Classifier-based scores are a standard approach for OOD detection due to their fine-grained detection capability. However, these scores often suffer from overconfidence issues, misclassifying OOD samples distant from the in-distribution region. To address this challenge, we propose a method called Nearest Neighbor Guidance (NNGuide) that guides the classifier-based score to respect the boundary geometry of the data manifold. NNGuide reduces the overconfidence of OOD samples while preserving the fine-grained capability of the classifier-based score. We conduct extensive experiments on ImageNet OOD detection benchmarks under diverse settings, including a scenario where the ID data undergoes natural distribution shift. Our results demonstrate that NNGuide provides a significant performance improvement on the base detection scores, achieving state-of-the-art results on both AUROC, FPR95, and AUPR metrics. The code is given at \url{https://github.com/roomo7time/nnguide}.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| out-of-distribution-detection-on-imagenet-1k-1 | NNGuide-ViM (ViT-B/16) | AUROC: 92.96 FPR95: 33.10 |
| out-of-distribution-detection-on-imagenet-1k-10 | NNGuide (RegNet) | AUROC: 95.82 FPR95: 17.00 Latency, ms: 31.00 |
| out-of-distribution-detection-on-imagenet-1k-10 | NNGuide (ResNet50 w/ ReAct) | AUROC: 96.11 FPR95: 17.27 Latency, ms: 11.10 |
| out-of-distribution-detection-on-imagenet-1k-11 | NNGuide (ResNet50 w/ ReAct) | AUROC: 92.49 FPR95: 35.1 Latency, ms: 11.10 |
| out-of-distribution-detection-on-imagenet-1k-11 | NNGuide (RegNet) | AUROC: 97.73 FPR95: 10.79 Latency, ms: 31.00 |
| out-of-distribution-detection-on-imagenet-1k-12 | NNGuide (RegNet) | AUROC: 95.42 FPR95: 17.97 |
| out-of-distribution-detection-on-imagenet-1k-12 | NNGuide (ResNet50 w/ ReAct) | AUROC: 95.45 FPR95: 19.72 |
| out-of-distribution-detection-on-imagenet-1k-3 | NNGuide (ResNet50 w/ ReAct) | AUROC: 97.7 FPR95: 11.12 Latency, ms: 11.10 |
| out-of-distribution-detection-on-imagenet-1k-3 | NNGuide (RegNet) | AUROC: 99.57 FPR95: 1.83 Latency, ms: 31.00 |
| out-of-distribution-detection-on-imagenet-1k-8 | NNGuide (RegNet) | AUROC: 94.43 FPR95: 21.58 |
| out-of-distribution-detection-on-imagenet-1k-9 | NNGuide (RegNet) | AUROC: 91.87 FPR95: 31.47 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.