Machine learning models can produce accurate/inaccurate predictions. However, their black box nature might prevent their easy adoption and enhancement by others. The SHAP [1]} is a popular model interpretation framework to interpret the classification/misclassification results of the model. In our experiment, we conduct a binary classification where the rejected edits was considered as the positive class and accepted edits as the negative class. Thus, our models attempt to predict the rejected edits by default. Therefore, a positive SHAP value indicates an increase in our models’ prediction of positive class and vice versa. Fig. REF  shows the importance of our selected features using a bee swarm plot from our random forest model. The bee swarm plot visualizes the SHAP value of a feature from each of the training instances on the x-axis. On the y-axis, it sorts all features in descending order according to their sum of SHAP values. The blue color indicates a low feature value, whereas the red indicates a high feature value in our plot. We see that reputation is the most important feature according to our random forest model. That is, community trust, which is estimated by reputation score, is an important predictor of the acceptability of suggested edits. We note that this feature with true response often leads to negative SHAP values, which indicates an increased prediction towards edit acceptance. That is, edits suggested by users with a high reputation score have a higher chance of being accepted and vice versa.
We also analyze our dataset for further insights on this. In particular, we conduct a comparative analysis of the reputation scores between users with rejected and accepted edits. We find the median reputation score of users whose suggested edits were rejected is 3428. On the contrary, such a score of users whose suggested edits were accepted is more than double (i.e., 7660). We then attempt to see whether the difference is statistically significant or not. We use the Mann-Whitney-Wilcoxon statistical significance test and find a statistical significance p-value (p-value \(\simeq \)  \(0.0\)  \(<\)  \(0.05\) ). We also use Cliff's delta test to determine the effect size and find a medium effect size (Cliff's d \(= -0.34\)  (medium)) with \(95\%\)  confidence. Given this evidence, low-reputed users are less trusted by the community, and thus their edits can be rejected.
