The documents from RVL-CDIP-O do not belong to any of the 16 RVL-CDIP categories, and we do not train the models on a 17th catch-all unknown category for out-of-domain documents.
Therefore, we benchmark two decision functions that use the model's output logits as a way to arbitrate between in- and out-of-domain predictions.
The first, termed Maximum Softmax Probability (MSP) obtains confidence scores from applying the softmax function on the logits (as done in e.g. [1]}, [2]}, [3]}).
If the model \(f\)  applied to a document image \(x\)  produces logits \(f(x) = z\) , then MSP applies the softmax function to obtain confidence scores scaled between 0 and 1 for each in-domain category. That is, the \(i^{th}\)  document category has a confidence score of
\(s(z)_i = \frac{e^{z_i}}{\sum _{j=1}^{K} e^{z_j}},\) 
