<jupyter_start><jupyter_text>Road Following - Data Collection (Mouse)If you've run through the collision avoidance sample in Day 2, you should now be familiar with these four steps:1. Data collection2. Training3. Optimizing3. DeploymentIn this notebook, we'll do the exact same thing! Except, instead of classification, you'll learn a different fundamental technique known as **regression**, which we'll use to enable our Jetbot to follow a path on the road.This notebook is for **Data Collection** only and we will be performing the following tasks:1. Display the live camera feed using a special ipywidget called jupyter clickable image widget2. Then collect data by performing the following steps:- Place the JetBot at different positions on a path (offset from center, different angles, etc)- Using your mouse, place a 'green dot', which corresponds to the target direction we want the robot to travel on.- Store the X, Y values of this green dot along with the image from the robot's cameraWhat we get, is a 'carrot on a stick' that moves along our desired trajectory.  Deep learning decides where to place the carrot, and JetBot just follows it :)  Install Dependencies!By default `jupyter_clickable_image_widget` is not installed on the Jetbot therefore we would need to first install it before continuing on with this notebook. To do that, go to your terminal and paste the following commands line by line:   1) cd   2) sudo apt-get install nodejs-dev node-gyp libssl1.0-dev   3) sudo apt-get install npm    4) git clone https://github.com/jaybdub/jupyter_clickable_image_widget   5) cd jupyter_clickable_image_widget   6) git checkout no_typescript   7) sudo pip3 install -e .   8) sudo jupyter labextension install js   9) sudo jupyter lab build     Import LibrariesSo lets get started by importing all the required libraries for "data collection" purpose. We will mainly use OpenCV to visualize and save image with labels. Libraries such as uuid, datetime are used for image naming.<jupyter_code># IPython Libraries for display and widgets
import ipywidgets
import traitlets
import ipywidgets.widgets as widgets
from IPython.display import display

# Camera and Motor Interface for JetBot
from jetbot import Robot, Camera, bgr8_to_jpeg

# Basic python packages for image annotation
from uuid import uuid1
import os
import json
import glob
import datetime
import numpy as np
import cv2
import time<jupyter_output><empty_output><jupyter_text>Collect dataFirstly, let's initialize and display our camera like we did in the previous notebooks, however, this time with using a special ipywidget called `jupyter_clickable_image_widget` that lets you click on the image and take the coordinates for data annotation. This eliminates the needs of using the gamepad controller for data annotation.Our neural network takes a 224x224 pixel image as input. We'll set our camera to that size to minimize the filesize of our dataset. In some scenarios it may be better to collect data in a larger image size and downscale to the desired size later.The following block of code below will display the live image feed for you to click on for annotation on the left, as well as the snapshot of last annotated image (with a green circle showing where you clicked) on the right.     When you click on the left live image, it stores a file in the ``dataset_xy`` folder with files named``xy___.jpg``When we train, we load the images and parse the x, y values from the filename.Here `` and `` are the coordinates **in pixels** (counted from the top left corner).<jupyter_code>from jupyter_clickable_image_widget import ClickableImageWidget

DATASET_DIR = 'dataset_xy'

# we have this "try/except" statement because these next functions can throw an error if the directories exist already
try:
    os.makedirs(DATASET_DIR)
except FileExistsError:
    print('Directories not created because they already exist')

camera = Camera()

# create image preview
camera_widget = ClickableImageWidget(width=camera.width, height=camera.height)
snapshot_widget = ipywidgets.Image(width=camera.width, height=camera.height)
traitlets.dlink((camera, 'value'), (camera_widget, 'value'), transform=bgr8_to_jpeg)

# create widgets
count_widget = ipywidgets.IntText(description='count')
# manually update counts at initialization
count_widget.value = len(glob.glob(os.path.join(DATASET_DIR, '*.jpg')))

def save_snapshot(_, content, msg):
    if content['event'] == 'click':
        data = content['eventData']
        x = data['offsetX']
        y = data['offsetY']
        
        # save to disk
        uuid = 'xy_%03d_%03d_%s' % (x, y, uuid1())
        image_path = os.path.join(DATASET_DIR, uuid + '.jpg')
        with open(image_path, 'wb') as f:
            f.write(camera_widget.value)
        
        # display saved snapshot
        snapshot = camera.value.copy()
        snapshot = cv2.circle(snapshot, (x, y), 8, (0, 255, 0), 3)
        snapshot_widget.value = bgr8_to_jpeg(snapshot)
        count_widget.value = len(glob.glob(os.path.join(DATASET_DIR, '*.jpg')))
        
camera_widget.on_msg(save_snapshot)

data_collection_widget = ipywidgets.VBox([
    ipywidgets.HBox([camera_widget, snapshot_widget]),
    count_widget
])

display(data_collection_widget)<jupyter_output><empty_output><jupyter_text>Now, let's close the camera connection properly so that we can use the camera in other notebooks.<jupyter_code>camera.stop()<jupyter_output><empty_output><jupyter_text>Save dataset in a zip (optional)Once you've collected enough data, we can choose to copy that data to a GPU desktop or cloud machine for training. To do that, we can call the following *terminal* command to compressour dataset folder into a single *zip* file.> The ! prefix indicates that we want to run the cell as a *shell* (or *terminal*) command.> The -r flag in the zip command below indicates *recursive* so that we include all nested files, the -q flag indicates *quiet* so that the zip command doesn't print any output<jupyter_code>!zip -r -q dataset_mouse.zip dataset_xy<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0671395579016826, 'fraction_numerical': 0.007093368525239195, 'mean_word_length': 4.645251396648045, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 19, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '656404', 'license': 'Apache-2.0', 'n_tokens_mistral': 1685, 'n_tokens_neox': 1639, 'n_words': 764, 'path': 'notebooks/Day_3/Road_Following/1-Data_collection-mouse.ipynb', 'repo_name': 'CertifaiAI/jetbot-labs', 'size': '6062'}