<commit_before># Assignment 3: Docker 
## Introduction
Docker is a way to wrap a solution with all it's dependencies in a container as a lightweight alternative to virtual machines and solving problems like dependency hell and deployment optimization. In order to ramp up and be able to package solutions the following will be needed:

## Complete by 9/29
### Dependencies
+ Make an account in [Docker Hub](https://hub.docker.com/)
+ We assume you have read the [Bash shell tutorial](http://www.ee.surrey.ac.uk/Teaching/Unix/) from assignment 0 or that you are familiar with the basic Unix commands.
+ Install Docker:
    - If you are working on Linux, Mac, or Windows Professional
    	+ Download the appropriate installer from [here](https://www.docker.com/products/docker#/windows) and follow the instructions.
    	+ Install Docker Compose from [here](https://docs.docker.com/compose/install/)
    - If you have a Windows Home installation, make sure to download Docker from [here](https://www.docker.com/products/docker-toolbox) and make sure to choose the option to install VirtualBox as Docker needs virtualization in non-Linux systems. Everytime that you need to execute Docker commands in the command line make sure to run *docker-machine start* to boot up the virtual machine in which Docker runs in your system and then run the appropriate [environment configuration command](https://getcarina.com/docs/tutorials/load-docker-environment-on-windows/). To make sure that it's setup correctly run *docker run hello-world*. In case it doesn't display 

    	> Hello from Docker!

    	There's a [troubleshooting](https://docs.docker.com/toolbox/faqs/troubleshoot/) section in the Docker documentation to configure the virtual machine.

+ Optional: Java, Python, and PiP are required if you want to follow the long example on how to use Docker.

### Readings
+ Work through the [Docker Documentation: Docker Overview](https://docs.docker.com/engine/understanding-docker/) to understand the terminology and the purpose of Docker. Also, work through the steps in [Get Started With Docker](https://docs.docker.com/engine/getstarted/step_two/) starting from 'Understanding images and containers' to 'Tag, push, & pull your image' to understand the basics of Docker. If you are running on Windows make sure you are running the MINGW console (either the one install by Git or the one installed through Docker) as some of the commands are not available in powershell (such as touch).
	From this you'll learn:
	- How Docker works
	- Docker's purpose
	- Containers and images
	- How to push and pull images from Docker hub
	- How to build images from dockerfiles
+ Read through [Docker Compose](https://docs.docker.com/compose/overview/) to understand how to run multiple containers. Read through Overview, installation, and 'getting started'.
+ Read [this article](https://gigaom.com/2014/06/10/why-companies-like-google-spotify-and-red-hat-are-embracing-dockers-open-source-containers/) to understand how the industry is using Docker to scale systems.
+ **OPTIONAL:** Read the [JHipster documentation](https://jhipster.github.io/docker-compose/) on how to package JHipster solution into a container. Read this only **after** the lecture on JHipster to better understand what is being done and what will be needed. You only have to read up to 'Building and running a Docker image of your application'. This helps you get an automated DockerFile. While in the project this will need to get done by hand, having an autogenerated file might help you get through.
+ **OPTIONAL:** [This](https://prakhar.me/docker-curriculum/#webapps) is a very thorough explanation on how to deploy a whole web app to Amazon using Docker and also explains best-practice usages of docker in real life development. Read through to understand usage how to use multiple containers using different techniques, port-forwarding, and how to search for existing images and services in Docker Hub and base your products off these. There's no need to follow the tutorial as many of the topics covered will also get covered by other groups, but reading it definitely explains how powerful containment technology is. Section 2 deploys a simple container to have a webapp running using flask in AWS. Section 3 explains how multi-container apps work in real life. 

## In-class assignment
### Pre-assignment
+ Quiz on the reading materials.
+ If you are using Docker-Toolset make sure you run `docker-machine start` and set the environment variables accordingly.
+ To verify you have downloaded everything before class, download and run our docker image by running `docker run -it --rm hoyosjs/installation_verification`. Basically you are fetching an image from DockerHub and runnining a container of it in interactive mode that gets deleted after it's used. 
+ Fill [this form](https://docs.google.com/a/vanderbilt.edu/forms/d/e/1FAIpQLScZ7tOaBJjbu95gM8l1jv4OmnYgagh8jVmrwxa1dukQd19xRA/viewform?c=0&w=1) with the output of the command.
+ Remove the image by running `docker rmi hoyosjs/installation_verification`

### Pack JHipster's team solution into a container
+ Create a file named `Dockerfile` with no extension.
+ Make sure you can get the `openjdk:8` image by running `docker search openjdk`
+ Create a Dockerfile on the folder containing your _JHipster_ solution.
+ As explained in the [reading material](https://docs.docker.com/engine/getstarted/step_four/) edit the `Dockerfile` to base your new image on `openjdk:8`. This image belongs to no specific user on docker hub so you don't have to specify a user name before it. 
+ Add a maintainer to the `Dockerfile` (I'd recommend using your email for this) using the `MAINTAINER` command.
+ Change the working directory with [`WORKDIR`](https://docs.docker.com/engine/reference/run/#workdir) to `/app/jhipster-project`.
+ Add the `war` file generated by building your _JHipster_ project in [production mode](https://jhipster.github.io/production/) to the working dir using the `ADD` command. Your file is in `build/libs/`. The syntax od the `ADD` command is `ADD <source_folder> <destination_folder>`. 
+ Add a line `EXPOSE 8080` to expose port 8080 by default.
+ As the last line of your Dockerfile use `CMD` to run the war file generated by jhipster. To run this you'll have to call `java` with the `jar` flag. You should also add `--spring.profiles.active=dev,swagger` at the end of the command to ensure your app is in dev mode, otherwise the web service will fail because of not having a database linked to it. 
+ Build your image with `docker build -t <your_docker_hub_user>/jhipster-project .`
+ Run `docker run -P -it --rm <your_docker_hub_user>/jhipster-project` to run your app. If you pass the flag `-i` it will run in interactive mode (with `-t` it binds the STDIN), if you pass `-d` it runs in detached (daemon) mode. 
+ Kill the running container with `CTRL + C`.
+ You can try accessing it from your browser. First run `docker ps` and get the port it's running on. If you are running Docker native go to `localhost:<port_number>` otherwise run `docker-machine env` and copy the tcp address for your machine ignoring the port number, and on your browser go to `<tcp_address>:<Port_number>`. 
+ Push your image to the hub using `docker login` and `docker push`.

### Getting the grading dependency configured
+ Create a file called `vunet` and write your vunetid. Double check the spelling.
+ Create a new dockerfile called `dockerfile2` in some other folder.
+ Base it off `hoyosjs/grading_helper`
+ Change the `WORKDIR` to `/app/data/`
+ Add the vunet file to the container by adding the `ADD ./vunet .` to the Dockerfile.
+ As the running command (hint: using `CMD`), use `/bin/bash` to run `./grade.sh` (included in the image in the working directory).
+ Build the image by saying `grading_helper`.

### Docker compose with grading dependency 
+ Create a file `docker-compose.yml`<commit_msg>- Updates to the Dockerfile creation section<commit_after># Assignment 3: Docker 
## Introduction
Docker is a way to wrap a solution with all it's dependencies in a container as a lightweight alternative to virtual machines and solving problems like dependency hell and deployment optimization. In order to ramp up and be able to package solutions the following will be needed:

## Complete by 9/29
### Dependencies
+ Make an account in [Docker Hub](https://hub.docker.com/)
+ We assume you have read the [Bash shell tutorial](http://www.ee.surrey.ac.uk/Teaching/Unix/) from assignment 0 or that you are familiar with the basic Unix commands.
+ Install Docker:
    - If you are working on Linux, Mac, or Windows Professional
    	+ Download the appropriate installer from [here](https://www.docker.com/products/docker#/windows) and follow the instructions.
    	+ Install Docker Compose from [here](https://docs.docker.com/compose/install/)
    - If you have a Windows Home installation, make sure to download Docker from [here](https://www.docker.com/products/docker-toolbox) and make sure to choose the option to install VirtualBox as Docker needs virtualization in non-Linux systems. Everytime that you need to execute Docker commands in the command line make sure to run *docker-machine start* to boot up the virtual machine in which Docker runs in your system and then run the appropriate [environment configuration command](https://getcarina.com/docs/tutorials/load-docker-environment-on-windows/). To make sure that it's setup correctly run *docker run hello-world*. In case it doesn't display 

    	> Hello from Docker!

    	There's a [troubleshooting](https://docs.docker.com/toolbox/faqs/troubleshoot/) section in the Docker documentation to configure the virtual machine.

+ Optional: Java, Python, and PiP are required if you want to follow the long example on how to use Docker.

### Readings
+ Work through the [Docker Documentation: Docker Overview](https://docs.docker.com/engine/understanding-docker/) to understand the terminology and the purpose of Docker. Also, work through the steps in [Get Started With Docker](https://docs.docker.com/engine/getstarted/step_two/) starting from 'Understanding images and containers' to 'Tag, push, & pull your image' to understand the basics of Docker. If you are running on Windows make sure you are running the MINGW console (either the one install by Git or the one installed through Docker) as some of the commands are not available in powershell (such as touch).
	From this you'll learn:
	- How Docker works
	- Docker's purpose
	- Containers and images
	- How to push and pull images from Docker hub
	- How to build images from dockerfiles
+ Read through [Docker Compose](https://docs.docker.com/compose/overview/) to understand how to run multiple containers. Read through Overview, installation, and 'getting started'.
+ Read [this article](https://gigaom.com/2014/06/10/why-companies-like-google-spotify-and-red-hat-are-embracing-dockers-open-source-containers/) to understand how the industry is using Docker to scale systems.
+ **OPTIONAL:** Read the [JHipster documentation](https://jhipster.github.io/docker-compose/) on how to package JHipster solution into a container. Read this only **after** the lecture on JHipster to better understand what is being done and what will be needed. You only have to read up to 'Building and running a Docker image of your application'. This helps you get an automated DockerFile. While in the project this will need to get done by hand, having an autogenerated file might help you get through.
+ **OPTIONAL:** [This](https://prakhar.me/docker-curriculum/#webapps) is a very thorough explanation on how to deploy a whole web app to Amazon using Docker and also explains best-practice usages of docker in real life development. Read through to understand usage how to use multiple containers using different techniques, port-forwarding, and how to search for existing images and services in Docker Hub and base your products off these. There's no need to follow the tutorial as many of the topics covered will also get covered by other groups, but reading it definitely explains how powerful containment technology is. Section 2 deploys a simple container to have a webapp running using flask in AWS. Section 3 explains how multi-container apps work in real life. 

## In-class assignment
### Pre-assignment
+ Quiz on the reading materials.
+ If you are using Docker-Toolset make sure you run `docker-machine start` and set the environment variables accordingly.
+ To verify you have downloaded everything before class, download and run our docker image by running `docker run -it --rm hoyosjs/installation_verification`. Basically you are fetching an image from DockerHub and runnining a container of it in interactive mode that gets deleted after it's used. 
+ Fill [this form](https://docs.google.com/a/vanderbilt.edu/forms/d/e/1FAIpQLScZ7tOaBJjbu95gM8l1jv4OmnYgagh8jVmrwxa1dukQd19xRA/viewform?c=0&w=1) with the output of the command.
+ Remove the image by running `docker rmi hoyosjs/installation_verification`

### Pack JHipster's team solution into a container

Docker packages an application and its dependencies into a portable format, called a container image. To build an image, you must provide a Dockerfile, which is a recipe for configuring the image. An image can then be run by Docker to create a container. Images are the equivalent of "classes" in OO programming and containers are the equivalent of "object instances". 

Dockerfiles start with a base image that is being inherited from (FROM). The official Dockerfile to build an Ubuntu server with Java 8 is shown below. As you can see, the image inherits from dockerfile/ubuntu, which is an image that provides a basic Linux Ubuntu installation. Each line starts with an instruction to docker (usually in CAPS) followed by its arguments. The full list of instructions for Dockerfiles is available here: https://docs.docker.com/engine/reference/builder/.

Some of the key instructions that you should be aware of are:
  
  + RUN \<cmd\> - runs a command inside of the image to configure it 
  + WORKDIR \<path\> - sets the directory that will be the default with the container is run (e.g., if you typed "pwd" in a shell, this is the directory it would return if you launched the image as a container)
  + ENV \<variable=value\> - sets an environment variable that is present in container instances
  + ADD \<local path\> \<container path\> - copies the directory specified by "local path" from your machine to the container at "container path"
  + CMD \<some command\> - determines what command is executed when a container instance is run
  

```
# Pull base image.
FROM dockerfile/ubuntu

# Install Java.
RUN \
  echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
  add-apt-repository -y ppa:webupd8team/java && \
  apt-get update && \
  apt-get install -y oracle-java8-installer && \
  rm -rf /var/lib/apt/lists/* && \
  rm -rf /var/cache/oracle-jdk8-installer


# Define working directory.
WORKDIR /data

# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle

# Define default command.
CMD ["bash"]


```

Next, we are going to learn how to package a Spring Boot application generated by JHipster into a Docker container image. We will create a new image that has a JVM, add the generated JAR file for the JHipster application to the image, and then specify the command to launch the JHipster application with the JVM.

If you completed the JHipster assignment, to create your own Dockerfile, do this:

+ Inside of the directory for your JHipster project (e.g., the folder containing build.gradle), create a file named `Dockerfile` with no extension inside of this directory

If you did not complete the JHipster assignment, do this:

+ Create a directory called docker-jhipster
+ Download the pre-built mock JHipster assignment binary to this directory: https://github.com/juleswhite/CSX278/raw/master/Assignments/Asgn3/jhipster-0.0.1-SNAPSHOT.war
+ Inside of the directory, create a file named `Dockerfile` with no extension 

Now, we need to find an existing image for our image to inherit from. To do this, we are going to use DockerHub (e.g., similar to GitHub but for Docker) to search for existing containers with a JDK. We will use one of the images that we find to inherit from in our "FROM" instruction within the Dockerfile.

+ Go to https://hub.docker.com/, search for "openjdk", find the official Open JDK image, and then copy the ID of the image (e.g., look for the command "docker pull \<image_id\>" -- you want just the ID after pull and not the "docker pull" part)
+ As explained in the [reading material](https://docs.docker.com/engine/getstarted/step_four/) edit the `Dockerfile` to base your new image on `openjdk:8`. This image belongs to no specific user on docker hub so you don't have to specify a user name before it. 

When you are done with these steps, you should have a basic Dockerfile with a FROM command specified. You can go ahead and test that your Dockerfile is correct by building it:

1. Open a terminal and change to the directory with your Dockerfile
2. Build your image with the command "docker build -t jdktest ./ " (note the "." at the end)

Once your image is built, it can now be run as a container. Let's run it.

1. In a terminal, run the command "docker run -it jdktest /bin/bash"
2. Your terminal propmp should change to something like "root@45e7a5e77784:/#" indicating that you are inside the container
3. Create a new directory called "test" (e.g., "mkdir test")
4. Make sure the direcotry is there "ls test"
5. Exit your container with the "exit" command
6. Check that the directory you added did not affect your local machine by running "ls" again and noticing that the directory is not on your local machine
7. Run the command "docker run -it jdktest /bin/bash" to run the image as a container again
8. Check if the directory that you created exists (it shouldn't)
9. Exit the container

You just ran a bare-bones container image that you created. When you launched the container and created the new directory, it was only added to the specific container instance that you were running. When you exited the container and relaunched it, all changes that you had made to it were lost since it was reset to its original state specified in the iamge.

Now, let's build out your Dockerfile to deploy the JHipster app:

+ Add a [`WORKDIR`](https://docs.docker.com/engine/reference/run/#workdir) instruction to the Dockerfile that changes the working directory to `/app/jhipster-project`.
+ Use the ADD command in the Dockerfile to add the `war` file for your _JHipster_ project (e.g., `jhipster-0.0.1-SNAPSHOT`) to the `/app/jhipster-project` dir. The syntax of the `ADD` command is `ADD <source_folder_or_file> <destination_folder_or_file>`. If you need to build your JHipster project, run "./gradlew bootRepackage". When the build completes, the war file is in `build/libs/`.
+ Add a maintainer to the `Dockerfile` (I'd recommend using your email for this) using the `MAINTAINER` command.

At this point, your Dockerfile should have "FROM", "ADD", "WORKDIR", and "MAINTAINER" instructions in it.

Let's test that we successfully added the WAR file and set the working directory for our image:

1. Build the container image by running "docker build -t jhipster-project ./"
2. Run a container from the image with "docker run -it jhipster-project /bin/bash"
3. Check that your working directory instruction was set correctly by running "pwd" and seeing that you are in `/app/jhipster-project`
4. Check that your `jhipster-0.0.1-SNAPSHOT.war` file is there
5. Check that you can run your jhipster app by typing `java -jar jhipster-0.0.1-SNAPSHOT.war --spring.profiles.active=dev,swagger`
6. When the JHipster logo shows up, kill the application with `control+c`
7. Exit the container

At this point, you have a working container image with the JHipster solution. However, we still can't access the JHipster web console from a browser yet. The reason for this is that Docker does not expose any networking ports on the container by default. We need to expose port 8080, which is the port that the JHipster application runs on. We also need to make our container automatically launch the JHipster application so that we don't have to login to a shell and run it manually.

Let's make the changes to expose the port that we need and run the JHipster application by default:

+ In your Dockerfile, add the line `EXPOSE 8080` to expose port 8080 by default.
+ On the last line of your Dockerfile use `CMD` to run the war file generated by jhipster. You will be specifying the same command that you ran manually (e.g., `java -jar jhipster-0.0.1-SNAPSHOT.war --spring.profiles.active=dev,swagger`) to launch the JHipster application. This instruction will make the container automatically run this command for you.

Let's build your container again and run it:

1. `docker build -t jhipster-project ./`
2. Now, we don't need to manually start the JHipster application and can map port 8080 to the container: `docker run -p 8080:8080 jhipster-project`
3. Wait until you see what is shown below and then open a browser to the "Local" address shown.

```
----------------------------------------------------------
	Application 'jhipster' is running! Access URLs:
	Local: 		http://127.0.0.1:8080
	External: 	http://172.17.0.2:8080
----------------------------------------------------------
```
4. To stop the container, use `CTRL + C`. 
5. If that doesn't work, open up another terminal and type "docker ps"
6. Look for a line like `0aaf5653efcc        jhipster-project    "/bin/sh -c 'java -ja"   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp   loving_noether`
7. Copy the ID that comes before "jhipster-project"
8. Run the command "docker stop \<ID\>

You now have a complete container image for your JHipster app. 

Let's review some of the differences in the run commands that we used. 

1. `docker run -it jhipster-project /bin/bash` runs the command "/bin/bash" inside of the container in order to log you into a shell within the container. Normally, you don't login to containers, but it can be useful for debugging or incrementally checking that a container is being built as you expect.
2. `docker run -d -p 8080:8080 jhipster-project` launches your container in daemon mode and maps port 8080 to port 8080 on the container. The `-d` flag specifies that the container is being run in daemon mode. The `-p 8080:8080` flag specifies that docker should map port 8080 on your local machine to port 8080 on the container. Since we aren't specifying a command after "jhipster-project", docker looks for our CMD instruction in the Dockerfile and runs it. 


### Getting the grading dependency configured
+ Create a file called `vunet` and write your vunetid. Double check the spelling.
+ Create a new dockerfile called `dockerfile2` in some other folder.
+ Base it off `hoyosjs/grading_helper`
+ Change the `WORKDIR` to `/app/data/`
+ Add the vunet file to the container by adding the `ADD ./vunet .` to the Dockerfile.
+ As the running command (hint: using `CMD`), use `/bin/bash` to run `./grade.sh` (included in the image in the working directory).
+ Build the image by saying `grading_helper`.

### Docker compose with grading dependency 
+ Create a file `docker-compose.yml`
{'fraction_non_alphanumeric': 0.07653235444766945, 'fraction_numerical': 0.009689260713675943, 'mean_word_length': 4.176535572249227, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 23, 'https://': 34, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 3, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1432763', 'n_tokens_mistral': 6623, 'n_tokens_neox': 6156, 'n_words': 3292}