<jupyter_start><jupyter_text>Test the pretrained modelGet the game screen as the input, and use the virtual joystick to steer the wheel.<jupyter_code>import sys
sys.path.append('../')
from europilot.screen import stream_local_game_screen
from europilot.screen import Box
from europilot.joystick import LinuxVirtualJoystick<jupyter_output><empty_output><jupyter_text>Load pretrained model, and check performance for each forward pass.The performance needs to be checked, because it determines the fps of our program.<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt

import os
import numpy as np
from PIL import Image
from time import time

import keras
from keras.models import load_model
from keras.preprocessing import image

parent_path = os.path.dirname(os.getcwd())
model_path = os.path.join(parent_path, 'model')

# multiply by constant to undo normalization
OUTPUT_NORMALIZATION = 655.35
# limit GPU memory usage
import tensorflow as tf
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.5
set_session(tf.Session(config=config))<jupyter_output><empty_output><jupyter_text>load pretrained model<jupyter_code>%ls $model_path
model = load_model(os.path.join(model_path, 'v5-PilotNet_v2-009-0.67221.h5'))<jupyter_output><empty_output><jupyter_text>Get sample image, to find out the performance of our model<jupyter_code>%ls ../sample/img/raw
sample_img = Image.open(os.path.join('../sample/img/raw', '9d0c3c2b_2017_07_27_14_55_08_16.jpg')).convert('RGB')
front_coord = (289,167, 851, 508)
sample_img_front = sample_img.crop(front_coord)
plt.imshow(sample_img_front)
sample_arr = image.img_to_array(sample_img_front)
sample_arr = np.reshape(sample_arr, (1,) + sample_arr.shape)
model.predict(sample_arr, batch_size = 1)
start = time()
for i in range(100):
    model.predict(sample_arr, batch_size = 1)
end = time()

fps = 100. / (end - start)
print("fps: %f" % fps)<jupyter_output>fps: 48.851024<jupyter_text>Be aware that the performance of the model can affect the performance of the overall program, since it may not react quickly enough to the changing environment.  Setup virtual joystick.Make sure to setup the virtual joystick inside ETS2 inside the controller settings. Get the coordinates for the game screen. The values may be different per game setting.<jupyter_code>x1, y1 = (68, 54)
x2, y2 = (x1 + 1024, y1 + 768)
box = Box(x1, y1, x2, y2)
joy = LinuxVirtualJoystick()<jupyter_output><empty_output><jupyter_text>As seen below, a virtual device can now be detected.<jupyter_code>!cat /proc/bus/input/devices<jupyter_output>I: Bus=0019 Vendor=0000 Product=0003 Version=0000
N: Name="Sleep Button"
P: Phys=PNP0C0E/button/input0
S: Sysfs=/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0E:00/input/input0
U: Uniq=
H: Handlers=kbd event0 
B: PROP=0
B: EV=3
B: KEY=4000 0 0

I: Bus=0019 Vendor=0000 Product=0001 Version=0000
N: Name="Power Button"
P: Phys=PNP0C0C/button/input0
S: Sysfs=/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input1
U: Uniq=
H: Handlers=kbd event1 
B: PROP=0
B: EV=3
B: KEY=10000000000000 0

I: Bus=0019 Vendor=0000 Product=0001 Version=0000
N: Name="Power Button"
P: Phys=LNXPWRBN/button/input0
S: Sysfs=/devices/LNXSYSTM:00/LNXPWRBN:00/input/input2
U: Uniq=
H: Handlers=kbd event2 
B: PROP=0
B: EV=3
B: KEY=10000000000000 0

I: Bus=0003 Vendor=046d Product=c29b Version=0111
N: Name="G27 Racing Wheel"
P: Phys=usb-0000:00:14.0-3/input0
S: Sysfs=/devices/pci0000:00/0000:00:14.0/usb1/1-3/1-3:1.0/0003:046D:C29B.0001/input/input6
U: Uniq=
H: Handlers=event3 js0 
B: PROP=0
[...]<jupyter_text>Set up game loop<jupyter_code># values computed from dataset sample.
def normalize(img):
    img[:,:,0] -= 89.5761
    img[:,:,0] /= 58.4214

    img[:,:,1] -= 97.5966
    img[:,:,1] /= 61.7917

    img[:,:,2] -= 88.3135
    img[:,:,2] /= 68.2043
    
    return img

def get_angle(predict):
    angle = predict[0][0]
    angle *= OUTPUT_NORMALIZATION
    
    return int(angle)
streamer = stream_local_game_screen(box=box, default_fps=60)
while True:
    image_data = next(streamer)
    im = Image.fromarray(image_data)
    img_front = im.crop(front_coord)
    
    arr = image.img_to_array(img_front)
    arr = normalize(arr)
    arr = np.reshape(arr, (1,) + arr.shape)
    
    angle = get_angle(model.predict(arr, batch_size = 1))
    
    joy.emit(angle)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11738250505959073, 'fraction_numerical': 0.07330784798740723, 'mean_word_length': 5.059945504087193, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '252499', 'license': 'MIT', 'n_tokens_mistral': 1898, 'n_tokens_neox': 1678, 'n_words': 400, 'path': 'scripts/05.inference.ipynb', 'repo_name': 'kc8055/europilot', 'size': '4447'}