Increasing the number of recognizable object classes is a widely studied problem in object recognition.
A typical approach in novel object recognition is to train a model that computes the similarity between the visual and semantic features of objects.
To compute semantic features of a novel object, external knowledge about the object (e.g., attributes [1]}, [2]}, [3]}, [4]}, [5]}, class hierarchy [6]}, [7]}, or textual description [8]}, [9]}, [10]}, [11]}) is often used.
Recently proposed vision-and-language contrastive learning methods, such as CLIP [12]} or ALIGN [13]}, use extremely large-scale image caption data to learn the relationship between images and their textual descriptions.
With the help of the prefix-tuning technique, these models exhibited a strong zero-shot recognition ability.
However, the abovementioned studies have a problem in that they require either a well-prepared knowledge database on novel objects or a large number of image-text pair datasets and appropriately designed prompts, both of which are labor-intensive tasks for humans.
In our method, once the question generation model is trained, the model dynamically acquires the necessary knowledge, thereby reducing human effort.
