Jiang Chengyue ; Hui Wenyang ; Jiang Yong ; Wang Xiaobin ; Xie Pengjun ; Tu Kewei

Abstract

Ultra-fine entity typing (UFET) predicts extremely free-formed types (e.g.,president, politician) of a given entity mention (e.g., Joe Biden) in context.State-of-the-art (SOTA) methods use the cross-encoder (CE) based architecture.CE concatenates the mention (and its context) with each type and feeds thepairs into a pretrained language model (PLM) to score their relevance. Itbrings deeper interaction between mention and types to reach better performancebut has to perform N (type set size) forward passes to infer types of a singlemention. CE is therefore very slow in inference when the type set is large(e.g., N = 10k for UFET). To this end, we propose to perform entity typing in arecall-expand-filter manner. The recall and expand stages prune the large typeset and generate K (K is typically less than 256) most relevant type candidatesfor each mention. At the filter stage, we use a novel model called MCCE toconcurrently encode and score these K candidates in only one forward pass toobtain the final type prediction. We investigate different variants of MCCE andextensive experiments show that MCCE under our paradigm reaches SOTAperformance on ultra-fine entity typing and is thousands of times faster thanthe cross-encoder. We also found MCCE is very effective in fine-grained (130types) and coarse-grained (9 types) entity typing. Our code is available at\url{https://github.com/modelscope/AdaSeq/tree/master/examples/MCCE}.