
ConceptMNIST: We augment the original MNIST dataset which already consists of images of various handwritten numbers from 0-9 along with their labels with carefully constructed, human understandable concepts to emulate real-world image recognition tasks. We propose to augment the dataset with a combination of overlapping and non-overlapping concepts as detailed below:

Non-Overlapping Concepts: Certain real-world concepts are shared by samples belonging to only a specific class instance - for example, in medical diagnosis of cancer, the presence of a specific malicious cell. We attempt to emulate such concepts by creating a vector of concepts which are specific to only a class of digits in MNIST. The easiest way to perform this is to one-hot encode labels for each handwritten digit to form a vector of size 10. For example for the number 4, the vector would be represented by [0,0,0,0,1,0,0,0,0,0] where each entry corresponds to a binary concept we call- “isNum\(_i\) ”. Note that no 2 digit classes can share the same vector representation.

Overlapping Concepts: Next, we emulate real-world concepts which are shared by samples from multiple classes - for example, in the CUB bird identification dataset, the color of the bird being black is shared by multiple bird classes. For MNIST, we consider 2 physical characteristics of numbers in the image - presence of a curved line and a presence of a straight line in their visualizations (the way they are printed in typeset formatting). We call these concepts as [“CurvedLine:present” , “StraightLine:present”] and are respectively set to 1 if the number can be constructed by only straight lines or curved lines in the standard typeset format. For example, the number 6 has no straight lines but only curved lines - hence is represented as [1,0] while the number 5 has both straight and curved lines and hence represented as [1,1]. Meanwhile, the number 7 has only straight lines so can be represented as [0,1]. Note that multiple digits can share similar representations, for example both numbers 0 and 6 will be represented by [1,0].

Both the “isNum\(_i\) ” and [“CurvedLine:present” , “StraightLine:present”] as described above are concatenated - resulting in a size 12 concept vector for each image. Note that this construction of concepts is completely up to user's subjective convenience and can be modified according to what the user thinks best interprets a prediction result. For a visual description of concept annotations, refer to Figure REF .

CUB: The Caltech-UCSD Birds-200-2011 dataset [1]} consists of 11,788 photos of 200 different classes of birds. The task is to classify each photo in one of the categories. The original dataset consists of 312 binary concepts representing physical traits. The dataset is processed exactly as described in [2]} - final dataset consists of 112 annotated concepts for each class of birds with each concept representing various physical traits of birds like wing color, beak size, etc. It is ensured all birds of same class contain same concept traits.

OAI: Osteoarthritis Initiative (OAI) dataset [3]} consists of X-ray images and clinical data for about 36,000 patients over 4 years of study who pose a risk of knee osteoarthritis. The task is to perform X-ray grading and classify each image into 4 different risk categories corresponding to Kellgren-Lawrence grade (KLG) which measures the severity of osteoarthritis, with higher numbers implying higher severity. Each image is annotated with 10 concepts which include medical diagnosis from X-ray images such as bone spacing. For a more comprehensive dataset details and processing description, refer to [2]} and [5]}. Note that OAI dataset is not publicly available and requires special permissions as detailed in [5]}.

<FIGURE>