Hollywood Identity Bias Dataset: A Context Oriented Bias Analysis of Movie Dialogues
Movies reflect society and also hold power to transform opinions. Social
biases and stereotypes present in movies can cause extensive damage due to
their reach. These biases are not always found to be the need of storyline but
can creep in as the author's bias. Movie production houses would prefer to
ascertain that the bias present in a script is the story's demand. Today, when
deep learning models can give human-level accuracy in multiple tasks, having an
AI solution to identify the biases present in the script at the writing stage
can help them avoid the inconvenience of stalled release, lawsuits, etc. Since
AI solutions are data intensive and there exists no domain specific data to
address the problem of biases in scripts, we introduce a new dataset of movie
scripts that are annotated for identity bias. The dataset contains dialogue
turns annotated for (i) bias labels for seven categories, viz., gender,
race/ethnicity, religion, age, occupation, LGBTQ, and other, which contains
biases like body shaming, personality bias, etc. (ii) labels for sensitivity,
stereotype, sentiment, emotion, emotion intensity, (iii) all labels annotated
with context awareness, (iv) target groups and reason for bias labels and (v)
expert-driven group-validation process for high quality annotations. We also
report various baseline performances for bias identification and category
detection on our dataset.