Finally, in Figure REF  we also observe OOD examples and adversarial examples. Adversarial examples [1]}, [2]}, [3]} refer to instances with small feature perturbations. A lot of studies [4]}, [5]}, [6]} use adversarial examples to evaluate and improve neural networks' robustness.
We can use diverse outliers to calibrate the model to output high uncertainty in the OOD region [7]}. Additionally, adversarial examples can be helpful to detect OOD examples close to ID regions.
Thus, our approach adopts a mixture of an auxiliary dataset of outliers and close adversarial examples to calibrate the predictive uncertainty.
We can easily get diverse text data as auxiliary outliers. However, generating adversarial examples via common gradient-based approaches is impossible in the NLP domain.
Thus, we apply methods [8]}, [9]}, [10]} to generate off-manifold adversarial examples from the embedding layer.
