Among the joint extraction approaches, HRL achieves significantly higher F1 scores on the two datasets. We run their model and our models five times and report the median results in Table REF . Scores of other baselines in Table REF  are taken from previous published papers [1]}, [2]}, [3]}. Our WordDecoding (WDec) model achieves F1 scores that are \(3.9\%\)  and \(4.1\%\)  higher than HRL on the NYT29 and NYT24 datasets respectively. Similarly, our PtrNetDecoding (PNDec) model achieves F1 scores that are \(3.0\%\)  and \(1.3\%\)  higher than HRL on the NYT29 and NYT24 datasets respectively. We perform a statistical significance test (t-test) under a bootstrap pairing between HRL and our models and see that the higher F1 scores achieved by our models are statistically significant (\(p < 0.001\) ). Next, we combine the outputs of five runs of our models and five runs of HRL to build ensemble models. For a test instance, we include those tuples which are extracted in the majority (\(\ge 3\) ) of the five runs. This ensemble mechanism increases the precision significantly on both datasets with a small improvement in recall as well. In the ensemble scenario, compared to HRL, WDec achieves \(4.2\%\)  and \(3.5\%\)  higher F1 scores and PNDec achieves \(4.2\%\)  and \(2.9\%\)  higher F1 scores on the NYT29 and NYT24 datasets respectively. When we ensemble the five runs of WDec and five runs of PNDec together, we see that it achieves the same performance as the ensemble version of WDec, since the tuples extracted by the ensemble version of the PNDec model are a proper subset of the tuples extracted by the ensemble version of the WDec model.
