The auxiliary datasets disjointed from the test datasets can be used to calibrate the neural networks' over-confidence for unseen samples.
A critical finding in [1]} is that the diversity of the auxiliary dataset is important. [2]} report that the methods using diverse examples beat the methods that only use close adversarial examples [3]}, [4]} in OOD detection in image classification.
Our empirical observations also find that randomly generated sentences (we randomly sample words and concatenate them into fake sentences) do not improve the performance. One partial explanation is that these "sentences" do not contain useful semantic information. This is similar to the CV domain, where CNN models do not extract valuable features from random pixel image samples.
Since it is easy to get a large corpus of diverse text data, utilizing a real dataset is inexpensive and straightforward.
Let \(P_{oe}({\bf \hat{x}})\)  be the distribution of the OOD auxiliary dataset, the regularization can be written as:
\(\max _{\Theta } \mathbb {E}_{{\bf \hat{x}} \sim P_{oe}({\bf \hat{x}})}[\text{Vac}(h \circ g({\bf \hat{x}}|\Theta ))]\) 
