TRAINING-and-EVALUATION.md

Training Steps

In this notebook, you'll need to follow 4 steps in order to train the model

1. Prepare prerequisite
2. Setup Paths
3. Train Model

Note: If you want to edit and add new codes to this notebook, I suggest clone the repo or save the notebook as a copy or else you'll lose your new codes.

Step 1 - Prepare Prerequisites

1.1 Clone my github repository

!git clone https://github.com/NyanSwanAung/Object-Detection-Using-DETR-CustomDataset.git
%cd Object-Detection-Using-DETR-CustomDataset/

1.2 Download face dataset from my github release page using wget

# Download train images
!wget https://github.com/NyanSwanAung/Object-Detection-Using-DETR-CustomDataset/releases/download/v1.0/WIDER_train.zip -O datasets/train.zip

# Download val images
!wget https://github.com/NyanSwanAung/Object-Detection-Using-DETR-CustomDataset/releases/download/v1.0/WIDER_val.zip -O datasets/val.zip

# Download annotations
!wget https://github.com/NyanSwanAung/Object-Detection-Using-DETR-CustomDataset/releases/download/v1.0/wider_face_split.zip -O datasets/annotations.zip

1.3 There is an already implemented dataloader and you just need to copy face dataloader to detr/datasets folder. You can modify the dataloader if you want.

!cp dataloaders/face.py /content/Object-Detection-Using-DETR-CustomDataset/detr/datasets 

# Make folder for trained-weights.
!mkdir detr/outputs

1.4 Install COCO API for evaluation

!pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'

The original Wider Face Dataset contains images and .mat for annotations. In order to train DETR, we need the dataset to be in COCO format below like this.

--annotations  # annotation json files
  |---- train.json
  |---- val.json
--train # train images
  |---- image0
  |---- image1
  |---- ....
--val      # val images
  |---- image0
  |---- image1
  |---- ....

1.5 Dataset Prepartion

There is a script datasets/face_to_coco.py which converts wider face dataset into COCO formatted dataset. Just simply run the cell and you will get train.json and val.json.

Before running the script, you need to set your dataset path in line 27 and 28 of the script.

25.      phases = ["train", "val"]
26.      for phase in phases:
27.         root_path = "datasets/WIDER_{}/images/".format(phase)
28.         gt_path = os.path.join("datasets/wider_face_split/wider_face_{}.mat".format(phase))

1.6 After setting up, run the script face_to_coco.py

!python datasets/face_to_coco.py

# Move json files to datasets folder
!mv train.json datasets && mv val.json datasets

After successful data preparation, you will get datasets folder like this

datasets
  |---- WIDER_train
  |---- WIDER_val
  |---- WIDER_test
  |---- wider_face_split
  |---- train.json
  |---- val.json

Step 2 - Setup Paths

DATASET_PATH = 'datasets'

TRAIN_IMG = 'datasets/WIDER_train/images'
TRAIN_JSON = 'datasets/train.json'

VAL_IMG = 'datasets/WIDER_val/images'
VAL_JSON = 'datasets/val.json'

Step 3 - Train Model

To train the model, you'll need to run detr/main.py from with required arguments. If you want to see supported arguments and the definition of arguments, run below code.

!python detr/main.py --help

If you want to train model with my configurations, run the code below.

!python detr/main.py \
    --batch_size=16 \
    --epochs=15 \
    --num_classes=2 \
    --num_queries=100 \
    --dataset_file='face' \
    --data_path={DATASET_PATH} \
    --train_folder={TRAIN_IMG} \
    --train_json={TRAIN_JSON} \
    --val_folder={VAL_IMG} \
    --val_json={VAL_JSON} \
    --output_dir='detr/outputs' \
    --resume={PRETRAINED_MODEL}

Evaluation Steps

To evaluate your model, you can use own trained weights or download my trained weights from github release page.

!wget https://github.com/NyanSwanAung/Object-Detection-Using-DETR-CustomDataset/releases/download/v1.0/detr_r50_ep15.tar

# Extract tar file
!tar -xf detr_r50_ep15.tar

Load DETR model from torch.hub and load ckpt file into model

TRAINED_CKPT_PATH = 'YOUR CHECKPOINT PATH' # sample --> checkpoint.pth
checkpoint = torch.load(TRAINED_CKPT, map_location='cpu')
model = torch.hub.load('facebookresearch/detr', 'detr_resnet50', pretrained=False, num_classes=2)
model.load_state_dict(checkpoint['model'], strict=False)

For simplicity, I've modified postprocessing function from Standalone Colab Notebook of original detr repo.

def postprocess_img(img_path): 
  im = Image.open(img_path)

  # mean-std normalize the input image (batch-size: 1)
  img = transform(im).unsqueeze(0)

  # propagate through the model
  start = time.time()
  outputs = model(img)
  end = time.time()
  print(f'Prediction time per image: {math.ceil(end - start)}s ', )

  # keep only predictions with 0.7+ confidence
  probas = outputs['pred_logits'].softmax(-1)[0, :, :-1]
  keep = probas.max(-1).values > 0.9

  # convert boxes from [0; 1] to image scales
  bboxes_scaled = rescale_bboxes(outputs['pred_boxes'][0, keep], im.size)
  
  plot_results(im, probas[keep], bboxes_scaled)

Load paths from test folder

TEST_IMG_PATH = 'datasets/WIDER_test/images'

img_format = {'jpg', 'png', 'jpeg'}
paths = list()

for obj in os.scandir(TEST_IMG_PATH):
  if obj.is_dir():
    paths_temp = [obj.path for obj in os.scandir(obj.path) if obj.name.split(".")[-1] in img_format]
    print()
    paths.extend(paths_temp)

Evaluate on first 10 images

for i in paths[1:10]:
  postprocess_img(i)

Metrics Visulaization

After training detr, you will get output folder containing log.txt. By using that output folder, you can visualize your metrics.

from detr.util.plot_utils import plot_logs
from pathlib import Path

log_directory = [Path('trained-weights')]

fields_of_interest = ('loss', 'mAP', 'loss_ce', 'loss_bbox', 'loss_giou', 'class_error', 'cardinality_error_unscaled')
plot_logs(log_directory, fields_of_interest)