A final reward score for model training is the summation of \(R_{aspect}\) and \(R_{taskAcc}\). Following Stiennon et al. (2020), we also incorporate penalties based on the Kullback Leibler (KL) divergence between the learned policy LM and \(M^{S}\).