Moreover, as shown in the last line of Table REF , it is clear that our Adaptive Distilling Attention (ADA) successfully boosts the performance, verifying the effectiveness of our approach. To further understand ADA’s ability of adaptively distilling useful prior and posterior knowledge, we summarize the average distilling weight values \(\lambda _1\)  and \(\lambda _2\)  according to the sentence type (normality and abnormality) in Table REF . Specifically, following [1]}, we consider sentences which contain “no”, “normal”, “clear”, “stable” as sentences describing normalities. As expected, the values of \(\lambda _1\)  and \(\lambda _2\)  generating the abnormalities are larger than the values generating the normalities. The reason is that both \(G^{\prime }_\text{Pr}\)  and \(W^{\prime }_\text{Pr}\)  contains much knowledge about the abnormalities, which indicates our ADA are capable of learning to efficiently distill the explored prior and posterior knowledge.
