<jupyter_start><jupyter_text>Utilisation des scriptsCe notebook vise à montrer l'utilisation des scripts du projet afin d'effectuer un entrainement et une inférence.<jupyter_code>%%shell

nvidia-smi<jupyter_output>Tue Aug 31 12:40:06 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |
| N/A   39C    P8    28W / 149W |      0MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-------[...]<jupyter_text>Installation du projet `bfc` et accès aux donnéesLes login et mot de passe sont nécessaires pour accéder à des repos privés à partir de Google/Colab. Cela ne serait pas le cas en accès public.<jupyter_code>import os
from getpass import getpass
user = getpass('GitHub user')
password = getpass('<PASSWORD>')
os.environ['GITHUB_AUTH'] = user + ':' + password
%%shell

git clone https://$GITHUB_AUTH@github.com/tibocour/IA.git<jupyter_output>Cloning into 'IA'...
remote: Enumerating objects: 86, done.[K
remote: Counting objects: 100% (86/86), done.[K
remote: Compressing objects: 100% (60/60), done.[K
remote: Total 86 (delta 36), reused 70 (delta 20), pack-reused 0[K
Unpacking objects: 100% (86/86), done.<jupyter_text>Installation des dépendances<jupyter_code>%%shell

pip install -r IA/requirements.txt<jupyter_output>Requirement already satisfied: imgaug in /usr/local/lib/python3.7/dist-packages (from -r IA/requirements.txt (line 1)) (0.2.9)
Collecting pascal-voc-writer
  Downloading pascal_voc_writer-0.1.4-py2.py3-none-any.whl (4.0 kB)
Collecting tflite-model-maker-nightly
  Downloading tflite_model_maker_nightly-0.3.4.dev202108310559-py3-none-any.whl (620 kB)
[K     |████████████████████████████████| 620 kB 5.4 MB/s 
[?25hRequirement already satisfied: pycocotools in /usr/local/lib/python3.7/dist-packages (from -r IA/requirements.txt (line 4)) (2.0.2)
Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from imgaug->-r IA/requirements.txt (line 1)) (1.4.1)
Requirement already satisfied: scikit-image>=0.11.0 in /usr/local/lib/python3.7/dist-packages (from imgaug->-r IA/requirements.txt (line 1)) (0.16.2)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from imgaug->-r IA/requirements.txt (line 1)) (4.1.2.30)
Requirement already sa[...]<jupyter_text>Apprentissage par transfer learning<jupyter_code>%%shell

python IA/python/train.py \
  --train_zip IA/data/train_megots150images.zip \
  --valid_zip IA/data/valid_megots150images.zip<jupyter_output>Tensorflow version : 2.6.0
args : Namespace(batch_size=10, epochs=50, train_zip='IA/data/train_megots150images.zip', valid_zip='IA/data/valid_megots150images.zip', with_tflite_eval=0)
created zip temporary directory: /tmp/tmpnfv4ogwe
extract zip file IA/data/train_megots150images.zip in /tmp/tmpnfv4ogwe/train
labels : ['megot']
extract zip file IA/data/valid_megots150images.zip in /tmp/tmpnfv4ogwe/valid
labels : ['megot']
* train dataset len : 1287
* valid dataset len : 30
2021-08-31 12:41:27.693597: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-31 12:41:28.340484: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-31 12:41:28.341471: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc[...]<jupyter_text>Inférence<jupyter_code>%%shell

rm -rf /tmp/images /tmp/annotations
unzip -q IA/data/valid_megots150images.zip -d /tmp

ls /tmp/images
%%shell

cd IA/python/google-coral-inference/

python detect_image.py \
  --model  /content/IA/data/efficientdet-lite-bfc.tflite \
  --labels /content/IA/data/bfc-labels.txt \
  --input  /tmp/images/1608644746.jpg \
  --output /content/1608644746_processed.jpg
from PIL import Image
im = Image.open("1608644746_processed.jpg")
import matplotlib.pyplot as plt
plt.imshow(im)
plt.show()
from google.colab import files
files.download("efficientdet-lite-bfc.tflite")
files.download("bfc-labels.txt")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.18167137666532096, 'fraction_numerical': 0.06035526846992329, 'mean_word_length': 2.85003885003885, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 22, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '559639', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1913, 'n_tokens_neox': 1673, 'n_words': 386, 'path': 'notebooks/train_tflite_model.ipynb', 'repo_name': 'tibocour/IA', 'size': '4959'}