There exists another way to retrieve this kind of data: to process tagged data with a conventional PoS tagger and analyze the resulting confusion matrix. We tested this approach by processing the whole BNC with the Stanford PoS Tagger [1]}. Note that as an input to the tagger we used not the whole sentences from the corpora, but separate tokens, to mimic our workflow with the distributional predictor. Prior to this, BNC tags were converted to the Penn Treebank tagsethttps://www.cis.upenn.edu/~treebank/ to match the output of the tagger. As we are interested in coarse, `overarching' word classes, inflectional forms were merged into one tag.
That was easy to accomplish by dropping all characters of the tags after the first two (excluding proper noun tags, which were all converted to NNP).
