@inproceedings{
ying2024dig,
title={{DIG}: Complex Layout Document Image Generation with Authentic-looking Text for Enhancing Layout Analysis},
author={Dehao Ying and Fengchang Yu and Haihua Chen and Wei Lu},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=xtnDvFtng3}
}
Even though significant progress has been made in standardizing document layout analysis, complex layout documents like magazines, newspapers, and posters still present challenges. Models trained on standardized documents struggle with these complexities, and the high cost of annotating such documents limits dataset availability. To address this, we propose the Complex Layout Document Image Generation (DIG) model, which can generate diverse document images with complex layouts and authentic-looking text, aiding in layout analysis model training. Concretely, we first pretrain DIG on a large-scale document dataset with a text-sensitive loss function to address the issue of unreal generation of text regions. Then, we fine-tune it with a small number of documents with complex layouts to generate new images with the same layout. Additionally, we use a layout generation model to create new layouts, enhancing data diversity. Finally, we design a box-wise quality scoring function to filter out low-quality regions during layout analysis model training to enhance the effectiveness of using the generated images. Experimental results on the DSSE-200 and PRImA datasets show when incorporating generated images from DIG, the mAP of the layout analysis model is improved from 47.05 to 56.07 and from 53.80 to 62.26, respectively, which is a 19.17% and 15.72% enhancement compared to the baseline.
DIG: Complex Layout Document Image Generation with Authentic-looking Text for Enhancing Layout Analysis
Dehao Ying Fengchang Yu Haihua Chen Wei Lu