In order to verify our claim that the entity embeddings from BERT better capture entity type information than those from [1]} ganea2017deep, we carry out an entity type prediction task based on its entity embedding. Specifically, we randomly sample 100K entities from Wikipedia, and randomly split them into training set (80K), development set (10K) and test set (10K). For each entity, we obtain its entity types from three typing systems: FIGER [2]}, BBN [3]} and OntoNotes\(_{\text{fine}}\)  [4]} via the entity type mapping provided by [5]} zhou-etal-2018-zero. The entity type prediction model is a simple linear classification modelDue to space limitation, we put the detailed description of this model and training hyper-parameters in the Appendices. using the entity embedding of an entity as features; limiting its capacity enables us
to focus on whether type information can be easily extracted from the entity embeddings.
We evaluate the model using standard entity typing metrics: Strict Accuracy (Acc.), Micro F1 (\(F1_{\text{mi}}\) ) and Macro F1 (\(F1_{\text{ma}}\) ).
