**Method 1 vs. Method 2**. We attempt to assess which one is more suitable for providing rewards for RL. For a fair comparison, we use the same evaluation data that was used in Method 2. Since Method 1 has 6 aspect categories, we obtain individual results using the best-performing approach and group them into two as we did for Method 2. Table 5 reports the accuracy, macro precision, macro recall, and macro F1 for Method 1 and Method 2. The results indicate that, although models from Method 2 are trained on a relatively small dataset, Method 2 is more aligned with human judgments compared to Method 1. Additionally, inference time using Method 2 is significantly faster than Method 1, making it easier to retrieve rewards in the RL setting. As a result, we use Method 2 for all our experiments.