Critics may point out that including trickier cases will inevitably lead to lower agreement, but I would answer that doing so should not decrease agreement on `easy cases', and that in instances of disagreement, most often each of the conflicting analyses corresponds to a valid reading of an ambiguous context (as demonstrated by [1]}). Human annotators also tend to disregard or not notice a relation more often than connecting non-coreferring entities [2]}, [3]}, meaning data would still have relatively high precision (for GUM, which includes the phenomena discussed here, student annotators achieved precision, recall and F1 of 0.918/0.811/0.861 compared to the adjudicated gold standard, when minimal mention span matching is used, [3]}). With this in mind, including these phenomena and labeling them as such, as is done in ARRAU or GUM, seems in essence already close to the ideal solution, with a few caveats and additions.
