In statistics,<unk>  (also called by various similar names, such as<unk><unk>  agreement,<unk><unk>  concordance,<unk> -observer<unk> ,<unk> -coder<unk> , and so on) is the degree of agreement among independent observers who<unk>  code, or assess the same phenomenon. Assessment tools that rely on ratings must exhibit good<unk> , otherwise they are not valid tests. There are a number of statistics that can be used to determine<unk> . Different statistics are appropriate for different types of measurement. Some options are joint-probability of agreement, such as Cohen's kappa, Scott's pi and Fleiss' kappa; or<unk><unk>  correlation, concordance correlation coefficient,<unk> a-class correlation, and Krippendorff's alpha.
Inter-rater reliability
Indicative limit value
Likelihood ratios in diagnostic testing
Proportional reporting ratio
Threshold limit value
Consensus error grid
Sensitivity and specificity
Predictive marker