<jupyter_start><jupyter_text>Autopilot - Basic EditionWe will use our trained model to make the jetBot move smoothly on the track.  Loading the modelWe assume that you have trained ``best_steering_model_xy.pth`` to the example folder in accordance with the instructions in the autopilot training model example.>  Execute the code below that we should be familiar with to initialize the PyTorch model.<jupyter_code>import torchvision
import torch

model = torchvision.models.resnet18(pretrained=False)
model.fc = torch.nn.Linear(512, 2)
from servoserial import ServoSerial
import threading
# Kill thpread
import inspect
import ctypes
import ipywidgets.widgets as widgets
from IPython.display import display
import time

controller = widgets.Controller(index=0)
display(controller)<jupyter_output><empty_output><jupyter_text>Load the trained model ``best_steering_model_xy.pth``.<jupyter_code>model.load_state_dict(torch.load('best_steering_model_xy.pth'))<jupyter_output><empty_output><jupyter_text>Because the model weights are on the CPU memory, then the following code is still passed to the GPU device as always.<jupyter_code>device = torch.device('cuda')
model = model.to(device)
model = model.eval().half()<jupyter_output><empty_output><jupyter_text>Create pre-processing functionIn order for the format of our training model to exactly match the format of the camera, we need to do some preprocessing.Proceed as follows:1. Convert from HWC layout to CHW layout2. Normalize using the same parameters as we did during training (our camera provides values in the range [0, 255], and the training loaded image is in the range [0, 1], so we need to scale 255.03. Transfer data from CPU memory to GPU memory4. Add a batch dimension<jupyter_code>import torchvision.transforms as transforms
import torch.nn.functional as F
import cv2
import PIL.Image
import numpy as np

mean = torch.Tensor([0.485, 0.456, 0.406]).cuda().half()
std = torch.Tensor([0.229, 0.224, 0.225]).cuda().half()

def preprocess(image):
    image = PIL.Image.fromarray(image)
    image = transforms.functional.to_tensor(image).to(device).half()
    image.sub_(mean[:, None, None]).div_(std[:, None, None])
    return image[None, ...]<jupyter_output><empty_output><jupyter_text>We have defined a preprocessing function that converts images from camera format to neural network input format.start showing our camera. After going through the last few examples, you should be familiar with this now.<jupyter_code>from IPython.display import display
import ipywidgets
import traitlets
from jetbot import Camera, bgr8_to_jpeg
from servoserial import ServoSerial

camera = Camera()
servo_device = ServoSerial() 

image_widget = ipywidgets.Image()

traitlets.dlink((camera, 'value'), (image_widget, 'value'), transform=bgr8_to_jpeg)

display(image_widget)

def camservoInitFunction():
    global leftrightpulse, updownpulse
    leftrightpulse = 2048
    updownpulse = 2048
    servo_device.Servo_serial_control(1, 2048)
    time.sleep(0.1)
    servo_device.Servo_serial_control(2, 1300)<jupyter_output><empty_output><jupyter_text>Create a robot instance to drive the motor<jupyter_code>from jetbot import Robot

robot = Robot()<jupyter_output><empty_output><jupyter_text>Define the slider to control the Jetbot.>  Tips: We have configured initial values for the sliders. These initial values apply to our official Yahboom map, but if you are training on your own different road maps, these values may not apply to your dataset, so Please increase or decrease the slider according to your settings and environment)1)Speed control (speed_gain_slider): To start Jetbot, add ``speed_gain_slider``2)Steering gain control (steering_gain_sloder): If you see that Jetbot is spinning, you need to reduce ``steering_gain_slider`` until it becomes smooth.3)Steering bias control (steering_bias_slider): If you see Jetbot leaning towards the far right or extreme left of the track, you should control this slider until Jetbot starts tracking the line or track at the center.>  Note: When you slide the related slider mentioned above, you should not move the slider value very quickly to get a smooth Jetbot road following behavior. You should adjust the motion parameter by moving the slider value gently.)<jupyter_code>speed_gain_slider = ipywidgets.FloatSlider(min=0.0, max=1.0, step=0.01, value=0.75,description='speed gain')
steering_gain_slider = ipywidgets.FloatSlider(min=0.0, max=1.0, step=0.01, value=0.33, description='steering gain')
steering_dgain_slider = ipywidgets.FloatSlider(min=0.0, max=0.5, step=0.001, value=0.12, description='steering kd')
steering_bias_slider = ipywidgets.FloatSlider(min=-0.3, max=0.3, step=0.01, value=0, description='steering bias')

display(speed_gain_slider, steering_gain_slider, steering_dgain_slider, steering_bias_slider)<jupyter_output><empty_output><jupyter_text>(Basic speed))speed_gain_slider     -> 0.51(Outer curve)   0.60(inner curve)    0.42    0.56(P)steering_gain_slider          -> 0.25           0.37            0.22    0.27(D)steering_dgain_slider         -> 0.24           0.24            0.10    0.13(Base steering value) steering_bias_slider -> 0.00   0.00   0.00    0.00 The x and y sliders will display the predicted x, y values. The steering slider will display our estimated steering value. This value is not the actual angle of the target, but an almost proportional value.When the actual angle is ``0``, this is 0, which will increase/decrease as the actual angle increases/decreases:<jupyter_code>x_slider = ipywidgets.FloatSlider(min=-1.0, max=1.0, description='x')
y_slider = ipywidgets.FloatSlider(min=0, max=1.0, orientation='vertical', description='y')
steering_slider = ipywidgets.FloatSlider(min=-1.0, max=1.0, description='steering')
speed_slider = ipywidgets.FloatSlider(min=0, max=1.0, orientation='vertical', description='speed')

display(ipywidgets.HBox([y_slider, speed_slider]))
display(x_slider, steering_slider)<jupyter_output><empty_output><jupyter_text>Create a function that will be called when the camera's value changes. This function will perform the following steps:1) Preprocess camera image2) Perform a neural network3) Calculate the approximate steering value4) Control the motor using proportional/differential control (PD)<jupyter_code>angle = 0.0
angle_last = 0.0

def execute(change):
    global angle, angle_last
    image = change['new']
    xy = model(preprocess(image)).detach().float().cpu().numpy().flatten()
    x = xy[0]
    y = (0.5 - xy[1]) / 2.0
    
    x_slider.value = x
    y_slider.value = y
    
    speed_slider.value = speed_gain_slider.value
    
    angle = np.arctan2(x, y)
    pid = angle * steering_gain_slider.value + (angle - angle_last) * steering_dgain_slider.value
    angle_last = angle

    steering_slider.value = pid + steering_bias_slider.value
    
    #PID+ Base speed + Gain
    robot.left_motor.value = max(min(speed_slider.value + steering_slider.value, 1.0), 0.0)
    robot.right_motor.value = max(min(speed_slider.value - steering_slider.value, 1.0), 0.0)
    
execute({'new': camera.value})<jupyter_output><empty_output><jupyter_text>We have created a neural network execution function, but now we need to attach it to the camera for processing.>  Tips: This code will make the robot move!! Please place the Jetbot robot on the map you have trained before. If the data you collected and the model are well trained, you will see Jetbot running smoothly on the road.<jupyter_code>camservoInitFunction()
camera.observe(execute, names='value')<jupyter_output><empty_output><jupyter_text>If your Jetbot functions properly, it will generate new commands for each new camera frame. Now you can place the Jetbot on a track that has collected data and see if it can track the track. If you want to stop this behavior, you can unload the binding of this callback function by executing the code in the following cell:<jupyter_code>camera.unobserve(execute, names='value')
time.sleep(0.1)
robot.stop()
robot.stop()
def jetbot_motion():
    count1 = count2 = count3 = count4 =  count5 = 0
    while 1:
        #Robot car left and right DC motor
        if controller.axes[1].value <= 0.1:
            if (controller.axes[0].value <= 0.1 and controller.axes[0].value >= -0.1 
                and controller.axes[1].value <= 0.1 and controller.axes[1].value >= -0.1):
                robot.stop()
            else:
                robot.set_motors(-controller.axes[1].value + controller.axes[0].value, -controller.axes[1].value - controller.axes[0].value)
            
            time.sleep(0.01)
        else:
            robot.set_motors(-controller.axes[1].value - controller.axes[0].value, -controller.axes[1].value + controller.axes[0].value)
            time.sleep(0.01)
          #Handle control code---2(Xbox360 Handle)
#         if controller.axes[1].value <= 0:
#             robot.set_motors(-controller.axes[1].value + controller.axes[0].value, -controller.axes[1].value - controller.axes[0].value)
#             time.sleep(0.01)
#         else:
#             robot.set_motors(-controller.axes[1].value - controller.axes[0].value, -controller.axes[1].value + controller.axes[0].value)
#             time.sleep(0.01)
            
# thread1 = threading.Thread(target=jetbot_motion)
# thread1.setDaemon(False)
# thread1.start()

def _async_raise(tid, exctype):
    """raises the exception, performs cleanup if needed"""
    tid = ctypes.c_long(tid)
    if not inspect.isclass(exctype):
        exctype = type(exctype)
    res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype))
    if res == 0:
        raise ValueError("invalid thread id") 
    elif res != 1:
        # """if it returns a number greater than one, you're in trouble,
        # and you should call it again with exc=NULL to revert the effect"""
        ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)
        
def stop_thread(thread):
    _async_raise(thread.ident, SystemExit)
thread1 = threading.Thread(target=jetbot_motion)
thread1.setDaemon(False)
thread1.start()
stop_thread(thread1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09809938626014651, 'fraction_numerical': 0.026133438922985548, 'mean_word_length': 4.422973698336017, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 55, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '244862', 'license': 'MIT', 'n_tokens_mistral': 3238, 'n_tokens_neox': 3059, 'n_words': 949, 'path': 'Autopilot-Basic/Autopilot-Basic.ipynb', 'repo_name': 'WyckliffeAluga/vulcan', 'size': '10102'}