Wasi Azmine Toushik ; Šerbetar Karlo ; Islam Raima ; Rafi Taki Hasan ; Chae Dong-Kyu

Abstract

In this paper, we introduce a framework ARBEx, a novel attentive featureextraction framework driven by Vision Transformer with reliability balancing tocope against poor class distributions, bias, and uncertainty in the facialexpression learning (FEL) task. We reinforce several data pre-processing andrefinement methods along with a window-based cross-attention ViT to squeeze thebest of the data. We also employ learnable anchor points in the embedding spacewith label distributions and multi-head self-attention mechanism to optimizeperformance against weak predictions with reliability balancing, which is astrategy that leverages anchor points, attention scores, and confidence valuesto enhance the resilience of label predictions. To ensure correct labelclassification and improve the models' discriminative power, we introduceanchor loss, which encourages large margins between anchor points.Additionally, the multi-head self-attention mechanism, which is also trainable,plays an integral role in identifying accurate labels. This approach providescritical elements for improving the reliability of predictions and has asubstantial positive effect on final prediction capabilities. Our adaptivemodel can be integrated with any deep neural network to forestall challenges invarious recognition tasks. Our strategy outperforms current state-of-the-artmethodologies, according to extensive experiments conducted in a variety ofcontexts.