The entity linking task is essentially a zero-shot task where the answer of test cases may not exist in the training data.Only 58.6% answers of test cases in AIDA-CoNLL dataset are existent in the training data. So we need to build a shared entity embedding space for all entities which allows neural entity linking models to generalize to both seen and unseen entities during test time.
Based on the distributional hypothesis [1]}, an entity is characterized by its contexts. Different methods to characterize an entity's context result in different information its entity embedding can capture. Previous work [2]}, [3]} on learning entity representation are mostly extensions of the embedding methods proposed by [4]}. An entity's context is a bag-of-words representation which mainly captures topic level entity relatedness rather than entity type relatedness.
In contrast, we propose a simple method to build entity embeddings directly from pre-trained BERT [5]} which can better capture entity type information.
