<commit_before>Using Jupyter Lab with Docker on Nvidia GPU instances
-----------------------------------------------------

Basic set up on Ubuntu
~~~~~~~~~~~~~~~~~~~~~~

Installing Docker CE (this follows `[1]`_)::

  # First, remove old versions of Docker
  sudo apt-get remove -y docker docker-engine docker.io

  # Next, add Docker Repository
  sudo apt-get update
  sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
  curl -fsSL 'https://download.docker.com/linux/ubuntu/gpg' | sudo apt-key add -
  sudo add-apt-repository \
      "deb [arch=amd64] https://download.docker.com/linux/ubuntu  $(lsb_release -cs) stable"

  # Install Docker CE
  sudo apt-get update
  sudo apt-get install -y docker-ce

  # Verify the Docker installation
  sudo docker run hello-world	

Installing Nvidia-Docker::

  # First, purge all of the old docker versions and containers.
  sudo docker volume ls -q -f driver=nvidia-docker \
      | xargs -r -I{} -n1 docker ps -q -a -f volume={} \
      | xargs -r docker rm -f
  sudo apt-get purge -y nvidia-docker

  # Add NVIDIA's docker repository to your system.
  # Install nvidia-docker2 and restart the Docker daemon.
  curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
      && curl -s -L "https://nvidia.github.io/nvidia-docker/$(. /etc/os-release; echo $ID$VERSION_ID)/nvidia-docker.list" \
      | sudo tee /etc/apt/sources.list.d/nvidia-docker.list \
      && sudo apt-get update \
      && sudo apt-get install -y nvidia-docker2 \
      && sudo pkill -SIGKILL dockerd

  # Test nvidia-smi within the Docker container.
  sudo docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi

Verifying GPU+TenosrFlow operation::

  docker run --gpus all -it --rm tensorflow/tensorflow:latest-gpu-py3-jupyter \
    python3 -c "import tensorflow as tf; print(tf.test.is_gpu_available())"

Basic Docker operation (for reference)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If the Docker daemon is not running, start it with::

  sudo service docker start

Pull the lastest Tensorflow image::

  docker pull tensorflow/tensorflow:latest-gpu-py3

To set up a custom repo for the image (effectively "forking" it)::

  docker tag <image-id> docker.io/<your-login-name>/<image-name>
  docker push docker.io/<your-ogin-name>/<image-name>

To see running images::

  docker ps -a

To commit and push changes::<commit_msg>Update docker-jupyter.rst<commit_after>Using Jupyter Lab with Docker on Nvidia GPU instances
-----------------------------------------------------

Basic set up on Ubuntu
~~~~~~~~~~~~~~~~~~~~~~

Installing Docker CE (this follows `[1]`_)::

  # First, remove old versions of Docker
  sudo apt-get remove -y docker docker-engine docker.io

  # Next, add Docker Repository
  sudo apt-get update
  sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
  curl -fsSL 'https://download.docker.com/linux/ubuntu/gpg' | sudo apt-key add -
  sudo add-apt-repository \
      "deb [arch=amd64] https://download.docker.com/linux/ubuntu  $(lsb_release -cs) stable"

  # Install Docker CE
  sudo apt-get update
  sudo apt-get install -y docker-ce

  # Verify the Docker installation
  sudo docker run hello-world	

Installing Nvidia-Docker::

  # First, purge all of the old docker versions and containers.
  sudo docker volume ls -q -f driver=nvidia-docker \
      | xargs -r -I{} -n1 docker ps -q -a -f volume={} \
      | xargs -r docker rm -f
  sudo apt-get purge -y nvidia-docker

  # Add NVIDIA's docker repository to your system.
  # Install nvidia-docker2 and restart the Docker daemon.
  curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
      && curl -s -L "https://nvidia.github.io/nvidia-docker/$(. /etc/os-release; echo $ID$VERSION_ID)/nvidia-docker.list" \
      | sudo tee /etc/apt/sources.list.d/nvidia-docker.list \
      && sudo apt-get update \
      && sudo apt-get install -y nvidia-docker2 \
      && sudo pkill -SIGKILL dockerd

  # Test nvidia-smi within the Docker container.
  sudo docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi

Verifying GPU+TenosrFlow operation::

  docker run --gpus all -it --rm tensorflow/tensorflow:latest-gpu-py3-jupyter \
    python3 -c "import tensorflow as tf; print(tf.test.is_gpu_available())"

Common Issues
~~~~~~~~~~~~~

I sometimes get an error after attempting to install ``nvidia-docker2``::

    The following packages have unmet dependencies:
     nvidia-docker2 : Depends: nvidia-container-runtime (>= 3.2.0) but 3.1.4-1 is to be installed
    E: Unable to correct problems, you have held broken packages.

That package requires a specific version of ``nvidia-container-runtime`` which in turn requires a
specific version of ``nvidia-container-toolkit``. You can view available versions of any package like so::

  apt-cache madison nvidia-container-toolkit
  
To fix the issue, I force installation of a specific version of ``nvidia-container-toolkit``::

  sudo apt-get install nvidia-container-toolkit=1.1.0-1
  
And then repeat the ``nvidia-docker2`` installation command.

Basic Docker operation (for reference)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If the Docker daemon is not running, start it with::

  sudo service docker start

Pull the lastest Tensorflow image::

  docker pull tensorflow/tensorflow:latest-gpu-py3

To set up a custom repo for the image (effectively "forking" it)::

  docker tag <image-id> docker.io/<your-login-name>/<image-name>
  docker push docker.io/<your-ogin-name>/<image-name>

To see running images::

  docker ps -a

To commit and push changes::
{'fraction_non_alphanumeric': 0.14290780141843973, 'fraction_numerical': 0.006382978723404255, 'mean_word_length': 3.129575402635432, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 14, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '888795', 'n_tokens_mistral': 1947, 'n_tokens_neox': 1704, 'n_words': 594}