Can human-generated annotations be incoherent? We did an experiment to verify whether if it is possible for humans to write incoherent descriptions. This is an important question because, if the human paragraphs are always coherent, then we can infer coherency of generated paragraphs by simply using scores such as BLEU. However, if human paragraphs are not always coherent, then relying only on scores such as BLEU is problematic and hence requires explicit coherence assessment methods like ours. We conducted a human study experiments where we asked people to describe 100 videos (randomly chosen from ActivityNet dataset ) without training them on how to write coherent paragraphs. We then evaluated the coherency of the descriptions using different set of human subjects. We found that about 72% of the descriptions are found to be incoherent [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}.
