In the backward pass, all the preference scores are updated with gradient descent method based on the straight-through gradient estimation [1]} using the validation set by propagating to \(s_{ji}\)  the gradients that \(w_{ji}\)  receives. Specifically, the selection function \(\rho _{s \rightarrow w}\)  is a kind of threshold function in Eq. (REF ), and we ignore the derivative of the function and pass on the gradients that \(w_{ji}\)  receives as if the function was an identity function. Therefore, we can bypass the selection function in the backward pass and compute the loss gradient with respect to \(s_{ji}\) :
\(\nabla _{s_{ji}}= \partial \mathcal {L}/\partial s_{ji}\approx \partial \mathcal {L} / \partial w_{ji}\) 
