RQ\({\mathbf {_3}}\) : Overall Relevance. Results show that RetroRank when using VSM+SA+TR achieved higher relevance than when the baseline VSM was used alone [1]}.The evidence is reported in Table REF . RetroRank (VSM+SA+TR) based on statistical significance reported by hypothesis H\(_6\) . Note that the obtained statistical test results are based on metrics defined in Section REF . A concrete example is also described in Table REF . Due to space limitations we only show results obtained by two participants. The key data in Table REF  is the `Relevance Score' column which shows the ranking scores from \([1-4]\)  as defined in Section REF . The relevance scores that participant P\(_{9}\)  in (group A) selected for RetroRank (VSM+SA+TR) were significantly higher than the relevance scores that participant P\(_9\)  in (group B) selected for VSM [1]}. We also observe a universal pattern across all bugs with the relevance scores from both approaches. For example, for the baseline VSM, the relevance scores tend to decrease within the top-10 query but for RetroRank (VSM+SA+TR) tend to increase within the top-10 query results. Another evidence found in this data is that the recommended comment that is placed in the first position is marked as `highly relevant'. However, we do not perceive a similar outcome from the results returned by the baseline VSM. Additionally, the query results from the baseline do not show a strong relationship relevant to bug's content in (group B). The query results from RetroRank (VSM+SA+TR) show a stronger relationship with the content from the query input and bug description. To illustrate, in all cases where relevance score is either 3 or 4, the results consist of vocabulary that refer to compiler signaling warnings. Hence, we pose that the differences in relevance of comments are statistically significant between RetroRank (VSM+SA+TR) and the baseline VSM. Such evidence is provided in Figure REF , hypothesise H\(_6\) , (\(4.5E-75\) ) with p-value of \(0.05\)  and a confidence level of \(0.95\) . The average relevance scores for RetroRank (VSM+SA+TR) (\(\mu =2.7\) ) are higher compared to the average relevance scores for the baseline VSM (\(\mu =1.4\) ).
