<jupyter_start><jupyter_text>Running This Guide: This guide is presented as a series of Jupyter notebooks covering both Tensorflow and PyTorch using a Python runtime.If you would like to run this code yourself, you can do so using the following steps: __Setup__:The basic prerequisites you will need to use TensorRT are:- a [supported NVIDIA GPU](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.htmlhardware-precision-matrix) (preferred compute capability 7.0 or higher - which includes INT8 precision support) - latest [NVIDIA GPU drivers](https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html)- a [supported CUDA and cuDNN](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.htmlplatform-matrix) installationYou can make sure your GPU environment is properly configured and check which GPU and CUDA version you are using with nvidia-smi:<jupyter_code>!nvidia-smi<jupyter_output><empty_output><jupyter_text>If you have Python versions 3.5 to 3.8 and CUDA 11.1 (or are in Google Colab), you can do the following to install TensorRT:<jupyter_code>!pip install --upgrade --index-url https://pypi.ngc.nvidia.com nvidia-tensorrt<jupyter_output><empty_output><jupyter_text>For some of the examples you will also need pycuda and onnx:<jupyter_code>pip install pycuda onnx<jupyter_output><empty_output><jupyter_text>__Tensorflow__:We will be using Tensorflow 2 to walk through the basic steps of deploying a TensorRT model. You can find Tensorflow installation instructions [here](), or you can use one of NVIDIA's NGC containers [here](https://ngc.nvidia.com/catalog/containers/nvidia:tensorflow). The Tensorflow examples also need keras2onnx and matplotlib:<jupyter_code>!pip install matplotlib keras2onnx<jupyter_output><empty_output><jupyter_text>__PyTorch__: We can also use other frameworks, such as PyTorch, with TensorRT by [exporting the model in ONNX format](https://pytorch.org/docs/stable/onnx.html). For this guide, we will demonstrate this process using PyTorch.You can find PyTorch installation instructions [here](https://pytorch.org/get-started/locally/), or use one of NVIDIA's NGC containers [here](https://ngc.nvidia.com/catalog/containers/nvidia:pytorch). You will also need torchvision:<jupyter_code>!pip install torchvision<jupyter_output><empty_output><jupyter_text>__Using Colab:__You can also test these notebooks out using [Google Colab](https://colab.research.google.com/), which includes both Tensorflow and PyTorch, as well as supported NVIDIA drivers. __Make sure to select a GPU hardware accelerator__ in the runtime options. Just note that TensorRT performance is best on newer gpus, Colab often has trouble with reduced precision inference, and you will have to use an older version of TensorRT.At the time of writing, Colab is on an old version of CUDA (10.1) that TensorRT 7 does not support. You will need to install TensorRT 6 - at the time of writing the following works to install it:<jupyter_code>%%bash

version="6.0.1-1+cuda10.1"
wget https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb

dpkg -i nvidia-machine-learning-repo-*.deb
apt-get update

sudo apt-get install libnvinfer6=${version} libnvonnxparsers6=${version} libnvparsers6=${version} libnvinfer-plugin6=${version} libnvinfer-dev=${version} libnvonnxparsers-dev=${version} libnvparsers-dev=${version} libnvinfer-plugin-dev=${version} python-libnvinfer=${version} python3-libnvinfer=${version}<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09050154104791258, 'fraction_numerical': 0.013449145418884841, 'mean_word_length': 6.183098591549296, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '167681', 'license': 'Apache-2.0', 'n_tokens_mistral': 1137, 'n_tokens_neox': 1076, 'n_words': 350, 'path': 'quickstart/IntroNotebooks/0. Running This Guide.ipynb', 'repo_name': 'boh-inspur/TensorRT', 'size': '3569'}