However, binarized selective rationalization schemes are notoriously hard to train [1]}, [2]}. To overcome training obstacles, previous works have considered using smoothed gradient estimations (e.g. gradient straight-through [3]} or Gumbel softmax [4]}), introducing additional components to control the complement of the selection [5]}, [2]}, adopting different updating dynamics between the generator and the predictor [7]}, using rectified continuous random variables to handle the constrained optimization in training [1]}, etc.
In practice, these solutions are still insufficient. They either still require careful tuning or are at a cost of reduced predictive accuracy.
