We can finally note that wrong or noisy labels, bad captioning (due to stereotyping, for instance) or the use of labeling algorithms that already contain bias (such as Natural Language Processing image interpreters) are also potential source of bias. An example of this phenomenon can be the subjective and socially biased choice of the attractive labels in the CelebA [1]} dataset. When image datasets include captioning as additional variable, the bias inherent to learned language model used to provide the caption is automatically included. For instance one of the main pre-trained algorithm in Natural Language Processing, Generative Pre-Trained Transformer 3, a.k.a GPT-3, is well known to be biased and thus generalized its bias to the image datasets as described for instance in [2]} for gender bias.
