<jupyter_start><jupyter_text>Part 4: Using GPU acceleration with PyTorch<jupyter_code># Execute this code block to install dependencies when running on colab
try:
    import torch
except:
    from os.path import exists
    from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag
    platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())
    cuda_output = !ldconfig -p|grep cudart.so|sed -e 's/.*\.\([0-9]*\)\.\([0-9]*\)$/cu\1\2/'
    accelerator = cuda_output[0] if exists('/dev/nvidia0') else 'cpu'

    !pip install -q http://download.pytorch.org/whl/{accelerator}/torch-1.0.0-{platform}-linux_x86_64.whl torchvision

try: 
    import torchbearer
except:
    !pip install torchbearer<jupyter_output>Collecting torchbearer
  Downloading torchbearer-0.5.3-py3-none-any.whl (138 kB)
[?25l
[K     |██▍                             | 10 kB 25.9 MB/s eta 0:00:01
[K     |████▊                           | 20 kB 22.8 MB/s eta 0:00:01
[K     |███████▏                        | 30 kB 11.3 MB/s eta 0:00:01
[K     |█████████▌                      | 40 kB 8.9 MB/s eta 0:00:01
[K     |███████████▉                    | 51 kB 5.3 MB/s eta 0:00:01
[K     |██████████████▎                 | 61 kB 5.8 MB/s eta 0:00:01
[K     |████████████████▋               | 71 kB 5.0 MB/s eta 0:00:01
[K     |███████████████████             | 81 kB 5.6 MB/s eta 0:00:01
[K     |█████████████████████▍          | 92 kB 5.7 MB/s eta 0:00:01
[K     |███████████████████████▊        | 102 kB 5.2 MB/s eta 0:00:01
[K     |██████████████████████████      | 112 kB 5.2 MB/s eta 0:00:01
[K     |████████████████████████████▌   | 122 kB 5.2 MB/s eta 0:00:01
[K     |██████████████████████████████▉ | 133 kB 5.2 MB/s eta 0:0[...]<jupyter_text>Manual use of `.cuda()`Now the magic of PyTorch comes in. So far, we've only been using the CPU to do computation. When we want to scale to a bigger problem, that won't be feasible for very long.|PyTorch makes it really easy to use the GPU for accelerating computation. Consider the following code that computes the element-wise product of two large matrices:<jupyter_code>import torch

t1 = torch.randn(1000, 1000)
t2 = torch.randn(1000, 1000)
t3 = t1*t2
print(t3)<jupyter_output>tensor([[-3.9552e-01, -3.8128e-01,  1.9382e-01,  ...,  6.5183e-02,
         -8.0798e-01, -1.2860e-01],
        [ 3.0110e-01, -8.0872e-02, -3.1637e-01,  ...,  1.3350e-02,
          1.6355e-03, -1.5694e+00],
        [-1.1489e-01,  7.3329e-01,  2.2787e-04,  ...,  3.7703e-01,
          4.1632e-01, -1.3523e+00],
        ...,
        [-7.3590e-02,  1.4592e+00,  1.3454e-01,  ...,  9.4198e-01,
         -1.8269e+00, -1.5354e-01],
        [-2.8336e-01, -2.4925e-01, -1.2408e-01,  ...,  7.4308e-01,
         -1.2395e+00,  8.6066e-01],
        [-5.7882e-01, -3.7655e-01,  1.0264e+00,  ...,  4.4065e-01,
          3.1151e-01,  2.3331e-01]])<jupyter_text>By sending all the tensors that we are using to the GPU, all the operations on them will also run on the GPU without having to change anything else. If you're running a non-cuda enabled version of PyTorch the following will throw an error; if you have cuda available the following will create the input matrices, copy them to the GPU and perform the multiplication on the GPU itself:<jupyter_code>t1 = torch.randn(1000, 1000).cuda()
t2 = torch.randn(1000, 1000).cuda()
t3 = t1*t2
print(t3)<jupyter_output>tensor([[ 7.8095e-01,  1.2063e+00, -4.9433e-01,  ...,  1.0070e+00,
         -1.3398e+00,  5.3305e-01],
        [-2.1063e-01, -8.1899e-01, -1.1687e-01,  ..., -1.5836e+00,
         -1.7853e-01, -7.4694e-01],
        [-1.3249e-01,  4.2734e-03, -1.0086e+00,  ...,  3.0373e-01,
          1.0454e-01, -2.2840e+00],
        ...,
        [ 5.8685e-02,  4.3033e-03,  7.7491e-01,  ..., -5.7385e+00,
          6.4804e-03, -1.0834e+00],
        [ 1.1656e-01,  6.8511e-01, -1.0661e-02,  ..., -4.6963e-01,
          3.1337e-01, -4.6921e-01],
        [ 4.3533e-01,  1.9739e-01,  7.2861e-02,  ..., -1.1576e-01,
         -4.2524e-03,  4.9364e-01]], device='cuda:0')<jupyter_text>If you're running this workbook in colab, now enable GPU acceleration (`Runtime->Runtime Type` and add a `GPU` in the hardware accelerator pull-down). You'll then need to re-run all cells to this point.If you were able to run the above with hardware acceleration, the print-out of the result tensor would show that it was an instance of `cuda.FloatTensor` type on the the `(GPU 0)` GPU device. If your wanted to copy the tensor back to the CPU, you would use the `.cpu()` method.  Writing platform agnostic codeMost of the time you'd like to write code that is device agnostic; that is it will run on a GPU if one is available, and otherwise it would fall back to the CPU. The recommended way to do this is as follows:<jupyter_code>device = "cuda:0" if torch.cuda.is_available() else "cpu"
t1 = torch.randn(1000, 1000).to(device)
t2 = torch.randn(1000, 1000).to(device)
t3 = t1*t2
print(t3)<jupyter_output>tensor([[ 7.5739e-03,  5.8514e-02,  1.3797e-01,  ...,  8.7184e-01,
          3.1220e-03,  7.8212e-01],
        [ 3.4573e-01,  4.4667e-01,  2.0699e-01,  ..., -7.2444e-01,
          3.4763e+00, -1.1593e+00],
        [-1.1349e+00, -7.0030e-01, -3.3237e-01,  ...,  8.4853e-02,
          3.9605e-01,  1.4900e-02],
        ...,
        [ 1.8884e-01,  3.8642e-01, -2.1536e-01,  ...,  6.1961e-01,
         -4.0988e-02, -7.9903e-02],
        [-2.3169e-01,  7.4789e-05,  5.5210e-01,  ...,  3.9327e-01,
         -8.5978e-01, -5.3543e-01],
        [-8.6620e-01, -1.9548e-01,  4.7630e-02,  ...,  8.4977e-01,
         -5.3451e-02, -1.2188e-01]], device='cuda:0')
{'fraction_non_alphanumeric': 0.19721577726218098, 'fraction_numerical': 0.1740139211136891, 'mean_word_length': 2.3199052132701423, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '230979', 'license': 'Apache-2.0', 'n_tokens_mistral': 3048, 'n_tokens_neox': 2407, 'n_words': 553, 'path': 'Pytorch Practical Tasks/4_4_GPU.ipynb', 'repo_name': 'VladimirsHisamutdinovs/deep-learning-pytorch', 'size': '5603'}