Inter-Annotator Agreement (IAA) measures the agreement between annotators while considering the possibility of chance agreement. A Cohen’s Kappa Coefficient [1]} of 85% for binary hope speech dataset and a Fleiss' Kappa Score [2]} of 82% for multiclass hope speech dataset indicate the strength of the datasets, which is the result of a strict annotation process.
