<jupyter_start><jupyter_text>Pytorch 추론시간 단축을 위한 정밀도 감소 - TensorRT / torch2trt library 환경설정- [Nvidia developer portal](https://developer.nvidia.com/nvidia-tensorrt-7x-download) 회원가입- TensorRT 7.0.0.11 for Ubuntu 1804 and CUDA 10.0 DEB local repo package 다운로드 후 colab 업로드 TensorRT 7.0 을 colab 에 설치하는 방법<jupyter_code>!nvcc --version<jupyter_output><empty_output><jupyter_text>colab CUDA 10.1 삭제<jupyter_code># remove CUDA 10.1

!sudo apt-get --purge remove cuda nvidia* libnvidia-*
!sudo dpkg -l | grep cuda- | awk '{print $2}' | xargs -n1 dpkg --purge
!sudo apt-get remove cuda-*
!sudo apt autoremove
!sudo apt-get update<jupyter_output><empty_output><jupyter_text>CUDA 10.0 설치 - 입력 세번 필요 (Y, 31, 1)<jupyter_code># Installing CUDA 10.0

!wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
!sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
!sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
!sudo apt-get update
!wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb
!sudo apt install -y ./nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb
!sudo apt-get update

# Install NVIDIA driver
!sudo apt-get -y installnvidia-driver-418

# Install development and runtime libraries (~4GB)
!sudo apt-get install -y \
    cuda-10-0 \
    libcudnn7=7.6.2.24-1+cuda10.0  \
    libcudnn7-dev=7.6.2.24-1+cuda10.0 --allow-change-held-packages
!nvcc --version<jupyter_output><empty_output><jupyter_text>TensorRT 설치 - colab .deb 파일 경로 확인<jupyter_code># install tensorrt
!sudo dpkg -i "/content/drive/My Drive/capstone1/CAN/nv-tensorrt-repo-ubuntu1804-cuda10.0-trt7.0.0.11-ga-20191216_1-1_amd64.deb"
!sudo apt-key add /var/nv-tensorrt-repo-cuda10.0-trt7.0.0.11-ga-20191216/7fa2af80.pub

!sudo apt-get update

!sudo apt-get install libnvinfer7=7.0.0-1+cuda10.0 libnvonnxparsers7=7.0.0-1+cuda10.0 libnvparsers7=7.0.0-1+cuda10.0 libnvinfer-plugin7=7.0.0-1+cuda10.0 libnvinfer-dev=7.0.0-1+cuda10.0 libnvonnxparsers-dev=7.0.0-1+cuda10.0 libnvparsers-dev=7.0.0-1+cuda10.0 libnvinfer-plugin-dev=7.0.0-1+cuda10.0 python-libnvinfer=7.0.0-1+cuda10.0 python3-libnvinfer=7.0.0-1+cuda10.0

!sudo apt-mark hold libnvinfer7 libnvonnxparsers7 libnvparsers7 libnvinfer-plugin7 libnvinfer-dev libnvonnxparsers-dev libnvparsers-dev libnvinfer-plugin-dev python-libnvinfer python3-libnvinfer

!sudo apt-get install tensorrt<jupyter_output><empty_output><jupyter_text>TensorRT 설치 확인<jupyter_code>!dpkg -l | grep TensorRT<jupyter_output><empty_output><jupyter_text>런타임 다시시작하기  [torch2trt](https://github.com/NVIDIA-AI-IOT/torch2trt) 라이브러리 설치<jupyter_code>cd /content/drive/My\ Drive/capstone1/CAN
!pip3 install pycuda
!git clone https://github.com/NVIDIA-AI-IOT/torch2trt
cd /content/drive/My\ Drive/capstone1/CAN/torch2trt
!python setup.py install
import tensorrt
import torch2trt<jupyter_output><empty_output><jupyter_text>추론시간 비교하기<jupyter_code>cd /content/drive/My\ Drive/capstone1/CAN
import utils
import numpy as np
import importlib
importlib.reload(utils)

utils.run_benchmark('./weights/fed_avg_50_0.9688.pth')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13249308330771595, 'fraction_numerical': 0.08453735013833384, 'mean_word_length': 5.093632958801498, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 5, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '249608', 'license': 'MIT', 'n_tokens_mistral': 1624, 'n_tokens_neox': 1537, 'n_words': 221, 'path': 'quantization/install_tensorRT.ipynb', 'repo_name': 'Jihunn-Kim/khu_capstone_1', 'size': '3253'}