Our baseline. We implement our baseline following [1]}, [2]}. Unlike our method that utilizes a unified transformer to deeply model interactions among the query words and the OCR words, our baseline models the interactions in a shallower way (see Section  for details). Besides, we also compare our method and baseline when initialized using different pre-trained models including Bert [3]}, LayoutLM [4]} and our SimpleDLM.
