Second-order word entropy has 3 hyperparameters: (i), the number of positively associated contexts \(N\)  to compute the average/median from; (ii), whether to use median or average entropy among the top \(N\)  contexts;Note that for any test pair, \(N\)  is the maximal number of associated contexts, which is reduced to \(M\)  if a test target has only \(M\)  (\(<N\) ) positively associated contexts in one of the two matrices to compare. and (iii), the association metric used to sort the contexts by relevance (i.e., PPMI or PLMI). We choose the following combination of hyperparameters: \(\langle \text{100, median, PLMI} \rangle \) , which is suggested by the work of [1]}.
