In order to evaluate the impact of our semi-supervisely trained actor-crtic reinforcement learning model over the task, we replace our actor-critic reinforcement learning model structure by a supervised BERT model with the same structure as the actor network, i.e. a LSTM structure with a hidden layer size of 200. The feedforward neural networks used to generate mention detection scores are kept as the same network with 150 units and ReLU activations. In order to train the model in a supervised manner, we use the reward \(r_t\)  as the coreference score or the training label for each given mention pairs fed into the system. The inputs of the training system are the mention representations of a document sample as in [1]}. The mentions scores and loss functions are kept as the same as in our actor-critic setup. During inference, mentions are clustered by using the generated coreference score in a given sample.
