Li Yuting ; Chen Yingyi ; Yu Xuanlong ; Chen Dexiong ; Shen Xi

Abstract

In this paper, we revisit techniques for uncertainty estimation within deepneural networks and consolidate a suite of techniques to enhance theirreliability. Our investigation reveals that an integrated application ofdiverse techniques--spanning model regularization, classifier andoptimization--substantially improves the accuracy of uncertainty predictions inimage classification tasks. The synergistic effect of these techniquesculminates in our novel SURE approach. We rigorously evaluate SURE against thebenchmark of failure prediction, a critical testbed for uncertainty estimationefficacy. Our results showcase a consistently better performance than modelsthat individually deploy each technique, across various datasets and modelarchitectures. When applied to real-world challenges, such as data corruption,label noise, and long-tailed class distribution, SURE exhibits remarkablerobustness, delivering results that are superior or on par with currentstate-of-the-art specialized methods. Particularly on Animal-10N and Food-101Nfor learning with noisy labels, SURE achieves state-of-the-art performancewithout any task-specific adjustments. This work not only sets a new benchmarkfor robust uncertainty estimation but also paves the way for its application indiverse, real-world scenarios where reliability is paramount. Our code isavailable at \url{https://yutingli0606.github.io/SURE/}.