**Knowledge Completion.** In knowledge completion tasks, LLMs might be (to some extent) considered as a knowledge base [576], which can be leveraged to complete or predict the missing parts of knowledge units (_e.g._, knowledge triples). Such tasks can probe and evaluate _how much_ and _what kind of_ knowledge LLMs have learned from the pre-training data. Existing knowledge completion tasks can be roughly divided into knowledge graph completion tasks (_e.g._, FB15k-237 [572] and WN18RR [574]) and fact completion tasks (_e.g._, WikiFact [571]), which aim to complete the triples from a knowledge graph and incomplete sentences about specific facts, respectively. Empirical studies have revealed that it is difficult for existing LLMs to accomplish knowledge completion tasks related to specific relation types [520]. As shown in the evaluation results on WikiFact, LLMs perform well on several frequent relations that occur in the pre-training data (_e.g._, currency and author), while not well on rare ones (_e.g._, discoverer_or_inventor and place_of_birth). Interestingly, under the same evaluation settings (_e.g._, in-context learning), InstructGPT (_i.e._, text-davinci-002) outperforms GPT-3 in all subsets of WikiFact.