<jupyter_start><jupyter_text>Introduction to Containerization with Docker  What is containerization? What are Containers?Containerization is the process of defining the software components required to run a *specific application*. In the context of a Docker-based containerization process this definition is written into a *Dockerfile*. A Dockerfile is then used to create a *Docker Image*. That Docker Image can then be used as the basis for one or more separate *Docker Containers*, or as the starting point for defining a new Dockerfile. Docker containers are immutable (i.e. changes to the container are **do not** update the source image, and are lost when the container is stopped *and* removed). A modified container can be used to create a new image through the *commit* command. ![Docker flow diagram](dockerFlow.png)This process of building a new image from a base image plus changes defined in a dockerfile, or changes made to a container based on that image and committed to a new image yields a model in which individual Docker images are made up of stacks of images that have been used to build previous base images. ![Docker layer model](layers.png) How is it different from virtualization (i.e. virtual machines)?You may have encountered references to *virtual machines* (or VMs) as a model for running virtual computer environments within/on computer hardware. A common question that arises when discussing containerization is how it differs from virtualization.  The primary difference between the two is that the machine images and corresponding virtual machines that they enable are fully self-contained virtual systems that provide all of the core capabilities (in virtual form) of a dedicated computer, including a full operating system (such as Windows or Liunx [Mac OS is infrequently used as an OS for VMs]) and set of applications running within that operating system. Containerized apps, on the other hand share a common container engine that provides core operating system capabilities that allow the application code within the container to run *as if* it were operating within a fully functional OS and hardware environment. ![Comparison of Hypervisor VM and Container Models](VMvsContainer.png)  Why would I use containers in my work?* Portability of **applications*** Comparatively lightweight* Consistent behavior across systems* Broad support    * Cloud providers        * [Google Cloud Platform](https://cloud.google.com/containers/)        * [Amazon Elastic Container Service](https://aws.amazon.com/ecs/)        * [Microsoft Azure Container Service](https://azure.microsoft.com/en-us/services/container-service/)    * Desktop operating systems (Docker Engine & additional tools - CE)        * [Mac OS](https://docs.docker.com/docker-for-mac/install/)        * [Windows](https://docs.docker.com/docker-for-windows/install/)    * Servers (CE)        * Linux             * [Ubuntu](https://docs.docker.com/install/linux/docker-ce/ubuntu/)            * [Debian](https://docs.docker.com/install/linux/docker-ce/debian/)            * [CentOS](https://docs.docker.com/install/linux/docker-ce/centos/)            * [Fedora](https://docs.docker.com/install/linux/docker-ce/fedora/)* You don't need to reinvent the wheel - Many pre-configured containers have been developed and maintained by the community     * There are numerous container images and documentation maintained in [Docker Hub](https://hub.docker.com) and [Docker Store](https://store.docker.com/search?q=&source=community&type=image) - You can share your images through Docker Hub as sell    * Many, many Dockerfiles maintained and accessible through [GitHub](https://github.com/search?utf8=✓&q=dockerfile&type=)* Containers can provide a foundation for a reproducable environment & workflow* Application containers are *imutable* and you can store data in data containers or through mounted directories on the host operating system* Multiple containers can be "composed" into combined services  Why would I *not* use containers?* Your workflows and applications are primarily Windows or Mac OS application based    * While you can run the Docker platform (i.e. Docker Engine) on a wide variety of operating systems, the applications that you can run in Docker containers are typically Linux based. * You typically work with applications for which there are robust installers for your particular operating system.   Sample Use Scenarios  Run a Debian Linux environment<jupyter_code>%%bash
# Pull the "Official" Debian docker image from Dockerhub 
docker pull debian<jupyter_output>Using default tag: latest
latest: Pulling from library/debian
Digest: sha256:4fcd8c0b6f5e3bd44a3e63be259fd0c038476d432953d449ef34aedf16def331
Status: Image is up to date for debian:latest<jupyter_text>Karls-MacBook-Pro-2:~ kbene$ docker run -it debianroot@e3d7e52268a5:/ cdroot@e3d7e52268a5:~ touch demo.txtroot@e3d7e52268a5:~ lsdemo.txtroot@e3d7e52268a5:~ exitexitKarls-MacBook-Pro-2:~ kbene$            Karls-MacBook-Pro-2:~ kbene$ docker run -it debianroot@5<PASSWORD>f5c490f:/ cdroot@5f4a7f5c490f:~ lsroot@5f4a7f5c490f:~ exitexitKarls-MacBook-Pro-2:~ kbene$   Setup and configure a web application<jupyter_code>%%bash
cd httpd
docker build -t kbene/httpd-local .
docker run -d -p 8081:80 --name httpd-local kbene/httpd-local
%%bash
docker ps<jupyter_output>CONTAINER ID        IMAGE               COMMAND              CREATED                  STATUS                  PORTS                  NAMES
d1e520feadab        kbene/httpd-local   "httpd-foreground"   Less than a second ago   Up Less than a second   0.0.0.0:8081->80/tcp   httpd-local<jupyter_text>[http://localhost:8081](http://localhost:8081)<jupyter_code>%%bash
docker stop httpd-local
docker rm httpd-local
docker ps<jupyter_output>httpd-local
httpd-local
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES<jupyter_text>Database server[MySQL Official Image](https://hub.docker.com/_/mysql/)<jupyter_code>%%bash
# Run the MySql server container from the official image in Dockerhub
docker pull mysql
docker run --name some-mysql -p 3306:3306 -e MYSQL_USER=demouser -e MYSQL_PASSWORD=<PASSWORD> -e MYSQL_ROOT_PASSWORD=<PASSWORD>-pw -e MYSQL_DATABASE=testdb -d mysql
%%bash
docker ps<jupyter_output>CONTAINER ID        IMAGE               COMMAND                  CREATED                  STATUS                  PORTS                    NAMES
e63d6212c683        mysql               "docker-entrypoint.s…"   Less than a second ago   Up Less than a second   0.0.0.0:3306->3306/tcp   some-mysql<jupyter_text>docker exec -it some-mysql bash    root@9098f791534c:/ mysql -u demouser -p    Enter password:     Welcome to the MySQL monitor.  Commands end with ; or \g.    Your MySQL connection id is 3    Server version: 5.7.21 MySQL Community Server (GPL)    Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.    Oracle is a registered trademark of Oracle Corporation and/or its    affiliates. Other names may be trademarks of their respective    owners.    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.    mysql> SHOW DATABASES;    +--------------------+    | Database           |    +--------------------+    | information_schema |    | testdb             |    +--------------------+    2 rows in set (0.00 sec)    mysql> QUIT;    Bye    root@9098f791534c:/<jupyter_code>%%bash
docker stop some-mysql
docker rm some-mysql
docker ps<jupyter_output>some-mysql
some-mysql
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
{'fraction_non_alphanumeric': 0.0804072575381804, 'fraction_numerical': 0.02480093982508811, 'mean_word_length': 3.4468949506674407, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 23, 'https://': 13, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '64590', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2245, 'n_tokens_neox': 2006, 'n_words': 828, 'path': '01-Docker Overview.ipynb', 'repo_name': 'unmrds/cc-containers', 'size': '7651'}