Given an image and a generated question, the Oracle Answerer predicts the answer knowledge to the question.
We implement this module as a composition of three sub-modules; (1) Head classifier, (2) Relation classifier, and (3) Region classifier.
Each module checks whether the generated question is “valid”, and if all modules agree that the question is “valid,” Oracle Answerer searches the oracle knowledge source and outputs the knowledge that matches the targeted head and relation.
Oracle knowledge source is a knowledge source that merges ConceptNet [1]} and Atomic\(^{20}_{20}\) [2]}, which were used in the creation of the datset (see Sec. REF ).
