<jupyter_start><jupyter_text>Reference: https://www.youtube.com/watch?v=sKhhb6RGv1c  Environment Sanity Check Click the _Runtime_ dropdown at the top of the page, then _Change Runtime Type_ and confirm the instance type is _GPU_.Check the output of `!nvidia-smi` to make sure you've been allocated a Tesla T4, P4, or P100.<jupyter_code>!nvidia-smi<jupyter_output>Mon Jul  5 01:27:25 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.27       Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
| N/A   41C    P8    10W /  70W |      0MiB / 15109MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-------[...]<jupyter_text>Setup:Set up script installs1. Updates gcc in Colab1. Installs Conda1. Install RAPIDS' current stable version of its libraries, as well as some external libraries including:  1. cuDF  1. cuML  1. cuGraph  1. cuSpatial  1. cuSignal  1. BlazingSQL  1. xgboost1. Copy RAPIDS .so files into current working directory, a neccessary workaround for RAPIDS+Colab integration.<jupyter_code># This get the RAPIDS-Colab install files and test check your GPU.  Run this and the next cell only.
# Please read the output of this cell.  If your Colab Instance is not RAPIDS compatible, it will warn you and give you remediation steps.
!git clone https://github.com/rapidsai/rapidsai-csp-utils.git
!python rapidsai-csp-utils/colab/env-check.py
# This will update the Colab environment and restart the kernel.  Don't run the next cell until you see the session crash.
!bash rapidsai-csp-utils/colab/update_gcc.sh
import os
os._exit(00)
# This will install CondaColab.  This will restart your kernel one last time.  Run this cell by itself and only run the next cell once you see the session crash.
import condacolab
condacolab.install()
# you can now run the rest of the cells as normal
import condacolab
condacolab.check()
# Installing RAPIDS is now 'python rapidsai-csp-utils/colab/install_rapids.py <release> <packages>'
# The <release> options are 'stable' and 'nightly'.  Leaving it blank or adding any other words will default to stable.
# The <packages> option are default blank or 'core'.  By default, we install RAPIDSAI and BlazingSQL.  The 'core' option will install only RAPIDSAI and not include BlazingSQL, 
!python rapidsai-csp-utils/colab/install_rapids.py stable
import os
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'
os.environ['CONDA_PREFIX'] = '/usr/local'<jupyter_output>Installing RAPIDS Stable 21.06
Starting the RAPIDS+BlazingSQL install on Colab.  This will take about 15 minutes.
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /usr/local

  added / updated specs:
    - cudatoolkit=11.0
    - gcsfs
    - llvmlite
    - openssl
    - python=3.7
    - rapids-blazing=21.06


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    abseil-cpp-20210324.1      |       h9c3ff4c_0        1015 KB  conda-forge
    aiohttp-3.7.4.post0        |   py37h5e8e339_0         625 KB  conda-for[...]<jupyter_text>cuDF and cuML Examples Now you can run code! What follows are basic examples where all processing takes place on the GPU. [cuDF](https://github.com/rapidsai/cudf)Load a dataset into a GPU memory resident DataFrame and perform a basic calculation.Everything from CSV parsing to calculating tip percentage and computing a grouped average is done on the GPU._Note_: You must import nvstrings and nvcategory before cudf, else you'll get errors.<jupyter_code>import cudf
import io, requests

import time # to record time

# download CSV file from GitHub
url="https://github.com/plotly/datasets/raw/master/tips.csv"
content = requests.get(url).content.decode('utf-8')
%%time
import cudf
import io, requests

import time # to record time

# download CSV file from GitHub
url="https://github.com/plotly/datasets/raw/master/tips.csv"
content = requests.get(url).content.decode('utf-8')

# start timer
start_time = time.time()

# read CSV from memory
tips_df = cudf.read_csv(io.StringIO(content))
tips_df['tip_percentage'] = tips_df['tip']/tips_df['total_bill']*100

# display average tip by dining party size
print(tips_df.groupby('size').tip_percentage.mean())

# end timer
end_time = time.time()
print("The total time to execute the code is {}".format(end_time - start_time))
%%time
import cudf
import io, requests

import time # to record time

# download CSV file from GitHub
url="https://github.com/plotly/datasets/raw/master/tips.csv"
content = requests.get(url).content.decode('utf-8')

# start timer
start_time = time.time()

# read CSV from memory
tips_df = cudf.read_csv(io.StringIO(content))
tips_df['tip_percentage'] = tips_df['tip']/tips_df['total_bill']*100

# display average tip by dining party size
print(tips_df.groupby('size').tip_percentage.mean())

# end timer
end_time = time.time()
print("The total time to execute the code is {}".format(end_time - start_time))<jupyter_output>size
6    15.622920
1    21.729202
4    14.594901
3    15.215685
2    16.571919
5    14.149549
Name: tip_percentage, dtype: float64
The total time to execute the code is 0.01599717140197754
CPU times: user 33.2 ms, sys: 3.23 ms, total: 36.5 ms
Wall time: 84.5 ms<jupyter_text>First time took longer since it has to load the libraries.<jupyter_code># cuDF type, not pandas type
type(tips_df)<jupyter_output><empty_output><jupyter_text>PandasCompare amount of time spent if using Pandas instead:<jupyter_code>%%time
import pandas as pd
import io, requests

import time # to record time

# download CSV file from GitHub
url="https://github.com/plotly/datasets/raw/master/tips.csv"
content = requests.get(url).content.decode('utf-8')

# start timer
start_time = time.time()

# read CSV from memory
tips_df = pd.read_csv(io.StringIO(content))
tips_df['tip_percentage'] = tips_df['tip']/tips_df['total_bill']*100

# display average tip by dining party size
print(tips_df.groupby('size').tip_percentage.mean())

# end timer
end_time = time.time()
print("The total time to execute the code is {}".format(end_time - start_time))<jupyter_output>size
1    21.729202
2    16.571919
3    15.215685
4    14.594901
5    14.149549
6    15.622920
Name: tip_percentage, dtype: float64
The total time to execute the code is 0.03443264961242676
CPU times: user 67.4 ms, sys: 19.1 ms, total: 86.4 ms
Wall time: 566 ms<jupyter_text>Using the standard Pandas, the amount of time processed is 0.034 seconds, as compared to cuDF's 0.015 seconds. Although the time saved is negligible, it would be significant if tested on larger datasets. [cuML](https://github.com/rapidsai/cuml)All calculations are performed on the GPU.<jupyter_code>%%time
import cuml

# start timer
start_time = time.time()

# Create and populate a GPU DataFrame
df_float = cudf.DataFrame()
# 3 features
df_float['0'] = [1.0, 2.0, 5.0]
df_float['1'] = [4.0, 2.0, 1.0]
df_float['2'] = [4.0, 2.0, 1.0]

# Setup and fit clusters
# dbscan clustering algo
dbscan_float = cuml.DBSCAN(eps=1.0, min_samples=1)
dbscan_float.fit(df_float)

print(dbscan_float.labels_)

# end timer
end_time = time.time()
print("The total time to execute the code is {}".format(end_time - start_time))
import time # to record time
import pandas as pd
%%time
from sklearn.cluster import DBSCAN

# start timer
start_time = time.time()

# Create and populate a GPU DataFrame
df_float = pd.DataFrame()
# 3 features
df_float['0'] = [1.0, 2.0, 5.0]
df_float['1'] = [4.0, 2.0, 1.0]
df_float['2'] = [4.0, 2.0, 1.0]

# Setup and fit clusters
# dbscan clustering algo
dbscan_float = DBSCAN(eps=1.0, min_samples=1)
dbscan_float.fit(df_float)

print(dbscan_float.labels_)

# end timer
end_time = time.time()
print("The total time to execute the code is {}".format(end_time - start_time))<jupyter_output>[0 1 2]
The total time to execute the code is 0.012969017028808594
CPU times: user 4.94 ms, sys: 967 µs, total: 5.9 ms
Wall time: 13 ms
{'fraction_non_alphanumeric': 0.1379764912666154, 'fraction_numerical': 0.04394155772822147, 'mean_word_length': 3.1438324988620847, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 8, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '418560', 'license': 'MIT', 'n_tokens_mistral': 3190, 'n_tokens_neox': 2921, 'n_words': 944, 'path': 'Notebooks/Simple_Demo_of_rapids_colab.ipynb', 'repo_name': 'Gamers-Blended/NVAITC', 'size': '9103'}