<jupyter_start><jupyter_text>Use LayoutParser to load and visualize COCO layout annotations  Preparation In this notebook, I will illustrate how to use LayoutParser to load and visualize the layout annotation in the COCO format. Before starting, please remember to download PubLayNet annotations and images from their [website](https://dax-cdn.cdn.appdomain.cloud/dax-publaynet/1.0.0/PubLayNet.html) (let's just use the validation set for now as the training set is very large). And let's put all extracted files in the `data/publaynet/annotations` and `data/publaynet/val` folder. And we need to install an additional library for conveniently handling the COCO data format:```bashpip install pycocotools```OK - Let's get on the code:   Loading and visualizing layouts using Layout-Parser<jupyter_code>from pycocotools.coco import COCO
import layoutparser as lp
import random 
import cv2
def load_coco_annotations(annotations, coco=None):
    """
    Args:
        annotations (List): 
            a list of coco annotaions for the current image 
        coco (`optional`, defaults to `False`):  
            COCO annotation object instance. If set, this function will 
            convert the loaded annotation category ids to category names 
            set in COCO.categories
    """
    layout = lp.Layout()

    for ele in annotations:

        x, y, w, h = ele['bbox']

        layout.append(
            lp.TextBlock(
                block = lp.Rectangle(x, y, w+x, h+y),
                type  = ele['category_id'] if coco is None else coco.cats[ele['category_id']]['name'],
                id = ele['id']
            )
        )
    
    return layout<jupyter_output><empty_output><jupyter_text>The `load_coco_annotations` function will help convert COCO annotations into the layoutparser objects.<jupyter_code>COCO_ANNO_PATH = 'data/publaynet/annotations/val.json'
COCO_IMG_PATH  = 'data/publaynet/val'

coco = COCO(COCO_ANNO_PATH)
color_map = {
    'text':   'red',
    'title':  'blue',
    'list':   'green',
    'table':  'purple',
    'figure': 'pink',
}


for image_id in random.sample(coco.imgs.keys(), 1):
    image_info = coco.imgs[image_id]
    annotations = coco.loadAnns(coco.getAnnIds([image_id]))
    
    image = cv2.imread(f'{COCO_IMG_PATH}/{image_info["file_name"]}')
    layout = load_coco_annotations(annotations, coco)
    
    viz = lp.draw_box(image, layout, color_map=color_map)
    display(viz) # show the results<jupyter_output><empty_output><jupyter_text>You could add more information in the visualization.<jupyter_code>lp.draw_box(image, 
              [b.set(id=f'{b.id}/{b.type}') for b in layout],
              color_map=color_map,
              show_element_id=True, id_font_size=10, 
              id_text_background_color='grey',
              id_text_color='white')<jupyter_output><empty_output><jupyter_text>Model Predictions on loaded data We could also check how the trained layout model performs on the input image. Following this [instruction](https://github.com/Layout-Parser/layout-parser/blob/master/examples/Deep%20Layout%20Parsing.ipynb), we could conveniently load a layout prediction model and run predictions on the existing image.<jupyter_code>model = lp.Detectron2LayoutModel('lp://PubLayNet/faster_rcnn_R_50_FPN_3x/config',
                                 extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.8],
                                 label_map={0: "text", 1: "title", 2: "list", 3:"table", 4:"figure"})
layout_predicted = model.detect(image)
lp.draw_box(image, 
              [b.set(id=f'{b.type}/{b.score:.2f}') for b in layout_predicted],
              color_map=color_map,
              show_element_id=True, id_font_size=10, 
              id_text_background_color='grey',
              id_text_color='white')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10453235525281634, 'fraction_numerical': 0.006811632171862719, 'mean_word_length': 3.294713160854893, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '152752', 'license': 'Apache-2.0', 'n_tokens_mistral': 1243, 'n_tokens_neox': 1186, 'n_words': 307, 'path': 'examples/Load and visualize layout annotations in the COCO format.ipynb', 'repo_name': 'rednam-ntn/layout-parser', 'size': '3817'}