Shaikh Mohammad Abuzar ; Duan Tiehang ; Chauhan Mihir ; Srihari Sargur

Abstract

The task of writer verification is to provide a likelihood score for whetherthe queried and known handwritten image samples belong to the same writer ornot. Such a task calls for the neural network to make it's outcomeinterpretable, i.e. provide a view into the network's decision making process.We implement and integrate cross-attention and soft-attention mechanisms tocapture the highly correlated and salient points in feature space of 2D inputs.The attention maps serve as an explanation premise for the network's outputlikelihood score. The attention mechanism also allows the network to focus moreon relevant areas of the input, thus improving the classification performance.Our proposed approach achieves a precision of 86\% for detecting intra-writercases in CEDAR cursive "AND" dataset. Furthermore, we generate meaningfulexplanations for the provided decision by extracting attention maps frommultiple levels of the network.