<commit_before>== GCP-jupyter-notebook
.This ansible script provisions a machine on the Google Cloud Platform (GCP) with:
* Oracle Java 8
* spark_version: "2.0.2-bin-hadoop2.7"
* Anaconda3-4.3.1-Linux-x86_64.sh (python3)
* Jupyter Scala binaries for Scala 2.11 when using python3
* The Anacoda https://anaconda.org/r/r-essentials[R Essentials bundle]

=== License
The MIT License (MIT).  See LICENSE.adoc

=== How to run
.To get this project working, perform the following steps:
----
# Step 1 (in your local machine)
# create the jupyter machine
$ gcloud compute instances create jupyter --scopes compute-rw --image-family ubuntu-1604-lts --image-project ubuntu-os-cloud --zone us-central1-f --scopes https://www.googleapis.com/auth/devstorage.read_write --machine-type n1-standard-1

# Step 2 (in your local machine)
# SSH into the jupyter machine
$ gcloud compute ssh jupyter --zone us-central1-f

# Step 3 (in the jupyter VM)
# clone repo
$ git clone https://github.com/yencarnacion/GCP-jupyter-notebook

# Step 4 (in the jupyter VM)
# provision the machine
$ cd GCP-jupyter-notebook 
$ sudo bash go.sh

# Step 5 (in the jupyter VM)
# get the jupyter notebook login token
$ sudo su ubuntu
$ /usr/lib/anaconda/bin/jupyter notebook list

# Step 6 (in your local machine)
# See Securely Connecting to VM Instances
# Forward ports from VM to local machine
# https://cloud.google.com/solutions/connecting-securely
$ gcloud compute ssh jupyter \
    --zone us-central1-f \
    --ssh-flag="-L" \
    --ssh-flag="8888:localhost:8888" \
    --ssh-flag="-N" 
----

.To Use:
* open `http://localhost:8888?token=<your token>` in your local machine browser
* change the directory to /home/ubuntu in the Jupiter Notebook

.Notes:
* Last run successfully on March 20, 2017
* You need the GCP IAM 'Compute Instance Admin' and the 'Storage Object Admin' permissions
* remember to do `gcloud compute instances delete jupyter --zone us-central1-f` when you are done<commit_msg>Added notes about R to README.adoc<commit_after>== GCP-jupyter-notebook
.This ansible script provisions a machine on the Google Cloud Platform (GCP) with:
* Oracle Java 8
* spark_version: "2.0.2-bin-hadoop2.7"
* Anaconda3-4.3.1-Linux-x86_64.sh (python3)
* Jupyter Scala binaries for Scala 2.11 when using python3
* The Anacoda https://anaconda.org/r/r-essentials[R Essentials bundle]

=== License
The MIT License (MIT).  See LICENSE.adoc

=== How to run
.To get this project working, perform the following steps:
----
# Step 1 (in your local machine)
# create the jupyter machine
$ gcloud compute instances create jupyter --scopes compute-rw --image-family ubuntu-1604-lts --image-project ubuntu-os-cloud --zone us-central1-f --scopes https://www.googleapis.com/auth/devstorage.read_write --machine-type n1-standard-1

# Step 2 (in your local machine)
# SSH into the jupyter machine
$ gcloud compute ssh jupyter --zone us-central1-f

# Step 3 (in the jupyter VM)
# clone repo
$ git clone https://github.com/yencarnacion/GCP-jupyter-notebook

# Step 4 (in the jupyter VM)
# provision the machine
$ cd GCP-jupyter-notebook 
$ sudo bash go.sh

# Step 5 (in the jupyter VM)
# get the jupyter notebook login token
$ sudo su ubuntu
$ /usr/lib/anaconda/bin/jupyter notebook list

# Step 6 (in your local machine)
# See Securely Connecting to VM Instances
# Forward ports from VM to local machine
# https://cloud.google.com/solutions/connecting-securely
$ gcloud compute ssh jupyter \
    --zone us-central1-f \
    --ssh-flag="-L" \
    --ssh-flag="8888:localhost:8888" \
    --ssh-flag="-N" 
----

.To Use:
* open `http://localhost:8888?token=<your token>` in your local machine browser
* change the directory to /home/ubuntu in the Jupiter Notebook

=== Notes about Anaconda R
----
# The R installed by Anaconda can be found at:
/usr/lib/anaconda/lib/R/bin/R
# it is different from the system installed R

# If you want to install R packages, you may want to open an R notebook and execute:
system("mkdir -p ~/anaconda3/lib/R/library", intern=TRUE)
# Then, in the first cell put:
.libPaths("~/anaconda3/lib/R/library")
# You can then install R packages as follows:
install.packages("png","~/anaconda3/lib/R/library")
----

.Other Notes:
* Last run successfully on March 20, 2017
* You need the GCP IAM 'Compute Instance Admin' and the 'Storage Object Admin' permissions
* remember to do `gcloud compute instances delete jupyter --zone us-central1-f` when you are done
{'fraction_non_alphanumeric': 0.11369801084990959, 'fraction_numerical': 0.024638336347197108, 'mean_word_length': 3.7631862217438106, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6043927', 'n_tokens_mistral': 1583, 'n_tokens_neox': 1473, 'n_words': 516}