Language is increasingly being used to define rich visual recognition problems with supporting image collections sourced from the web . Structured prediction models are used in these tasks to take advantage of correlations between co-occurring labels and visual input but risk inadvertently encoding social biases found in web corpora . In this work , we study data and models associated with multilabel object classification and visual semantic role labeling . We find that ( a ) datasets for these tasks contain significant gender bias and ( b ) models trained on these datasets further amplify existing bias . For example , the activity cooking is over 33 % more likely to involve females than males in a training set , and a trained model further amplifies the disparity to 68 % at test time . We propose to inject corpus-level constraints for calibrating existing structured prediction models and design an algorithm based on Lagrangian relaxation for collective inference . Our method results in almost no performance loss for the underlying recognition task but decreases the magnitude of bias amplification by 47.5 % and 40.5 % for multilabel classification and visual semantic role labeling , respectively . Visual recognition tasks involving language , such as captioning ( Vinyals et al . , 2015 ) , visual question answering ( Antol et al . , 2015 ) , and visual semantic role labeling ( Yatskar et al . , 2016 ) , have emerged as avenues for expanding the diversity of information that can be recovered from images . These tasks aim at extracting rich seman-tics from images and require large quantities of labeled data , predominantly retrieved from the web . Methods often combine structured prediction and deep learning to model correlations between labels and images to make judgments that otherwise would have weak visual support . For example , in the first image of Figure 1 , it is possible to predict a spatula by considering that it is a common tool used for the activity cooking . Yet such methods run the risk of discovering and exploiting societal biases present in the underlying web corpora . Without properly quantifying and reducing the reliance on such correlations , broad adoption of these models can have the inadvertent effect of magnifying stereotypes . In this paper , we develop a general framework for quantifying bias and study two concrete tasks , visual semantic role labeling ( vSRL ) and multilabel object classification ( MLC ) . In vSRL , we use the imSitu formalism ( Yatskar et al . , 2016 ( Yatskar et al . , , 2017 ) ) , where the goal is to predict activities , objects and the roles those objects play within an activity . For MLC , we use MS-COCO ( Lin et al . , 2014 ; Chen et al . , 2015 ) , a recognition task covering 80 object classes . We use gender bias as a running example and show that both supporting datasets for these tasks are biased with respect to a gender binary1 . Our analysis reveals that over 45 % and 37 % of verbs and objects , respectively , exhibit bias toward a gender greater than 2:1 . For example , as seen in Figure 1 , the cooking activity in imSitu is a heavily biased verb . Furthermore , we show that after training state-of-the-art structured predictors , models amplify the existing bias , by 5.0 % for vSRL , and 3.6 % in MLC . Each image is paired with a table describing a situation : the verb , cooking , its semantic roles , i.e agent , and noun values filling that role , i.e. woman . In the imSitu training set , 33 % of cooking images have man in the agent role while the rest have woman . After training a Conditional Random Field ( CRF ) , bias is amplified : man fills 16 % of agent roles in cooking images . To reduce this bias amplification our calibration method adjusts weights of CRF potentials associated with biased predictions . After applying our methods , man appears in the agent role of 20 % of cooking images , reducing the bias amplification by 25 % , while keeping the CRF vSRL performance unchanged . To mitigate the role of bias amplification when training models on biased corpora , we propose a novel constrained inference framework , called RBA , for Reducing Bias Amplification in predictions . Our method introduces corpus-level constraints so that gender indicators co-occur no more often together with elements of the prediction task than in the original training distribution . For example , as seen in Figure 1 , we would like noun man to occur in the agent role of the cooking as often as it occurs in the imSitu training set when evaluating on a development set . We combine our calibration constraint with the original structured predictor and use Lagrangian relaxation ( Korte and Vygen , 2008 ; Rush and Collins , 2012 ) to reweigh bias creating factors in the original model . We evaluate our calibration method on imSitu vSRL and COCO MLC and find that in both instances , our models substantially reduce bias amplification . For vSRL , we reduce the average magnitude of bias amplification by 40.5 % . For MLC , we are able to reduce the average magnitude of bias amplification by 47.5 % . Overall , our calibration methods do not affect the performance of the underlying visual system , while substantially reducing the reliance of the system on socially biased correlations 2 . 2 Code and data are available at https://github . com / uclanlp / reducingbias Structured prediction models can leverage correlations that allow them to make correct predictions even with very little underlying evidence . Yet such models risk potentially leveraging social bias in their training data . In this paper , we presented a general framework for visualizing and quantifying biases in such models and proposed RBA to calibrate their predictions under two different settings . Taking gender bias as an example , our analysis demonstrates that conditional random fields can amplify social bias from data while our approach RBA can help to reduce the bias . Our work is the first to demonstrate structured prediction models amplify bias and the first to propose methods for reducing this effect but significant avenues for future work remain . While RBA can be applied to any structured predictor , it is unclear whether different predictors amplify bias more or less . Furthermore , we presented only one method for measuring bias . More extensive analysis could explore the interaction among predictor , bias measurement , and bias deamplification method . Future work also includes applying bias reducing methods in other structured domains , such as pronoun reference resolution ( Mitkov , 2014 ) .
Language is used for visual recognition problems such as captioning to improve performance however it can also encode social biases found in web corpora.
They propose a framework to quantify bias for visual semantic role labelling and multilabel object classification and a constraint inference framework to calibrate existing models.
They find that existing datasets contain gender bias the use of text can amplify it, and the proposed framework can reduce bias without performance loss.