Ideally, we want methods to flag new semantic content (irrespective of which dataset they belong to) since model predictions on such inputs cannot be trusted, while at the same time being able to identify and not flag semantically-similar inputs (even if they're from different datasets) as inputs that the model has generalized to. We also follow the setting recommended by [1]} to avoid dataset bias during training by holding out few classes from a dataset during training and using the held-out classes as new semantic content for evaluation (See Section REF ).
