We propose a pipeline consisting of a knowledge-based object classifier (OC) and a question generator (QG) for knowledge acquisition.
Following previous research on structured knowledge [1]}, we represent knowledge as a knowledge triplet, that is, a list of three words or phrases: head, relation, and tail, such as \(\langle \) dog, IsA, mammal\(\rangle \) .
We train the OC to retrieve knowledge from knowledge sources, which outputs the corresponding head in the knowledge source as the predicted object class (e.g., 〈IsA, mammal〉\(\,\rightarrow \,\) dog).
The QG model then generates questions to add new knowledge to the knowledge source for novel object recognition.
In the QG model, we use two modes in question generation: confirmation and exploration, as illustrated in Figure REF .
First, “confirmation” is used when the unknown object is relatively close to a known object category.
For example, if the model knows about “dog,” then a novel category “chihuahua” is considered to be a close concept to “dog.”
In this case, the model can infer reasonable knowledge (e.g., both “chihuahua” and “dog” are a type of mammal) and ask questions to confirm it, such as “What is the mammal on the left side of the image?”
In contrast, the “exploration” mode is used when the unknown object is far from the existing object category (e.g., “teddy-bear” may not resemble any known object class).
In this case, the model is unable to estimate the proper knowledge and attempts to obtain all the necessary knowledge by asking questions (“What is the object sitting next to the dog made of?”).
