In prior work, a fixed threshold (r t = 0.5) is used for classification of all types (Ling and Weld, 2012;<cite> Shimaoka et al., 2017)</cite> . We instead assign a different threshold to each type that is optimized to maximize the overall strict F 1 on the dev set.