<commit_before>User Documentation
==================

This user documentation contains information about how to schedule a task with CC-Server to run the CC-Sample-App and
how to build a custom Docker images containing arbitrary applications.

Docker Installation
-------------------

Please follow the official `Docker documenation <https://docs.docker.com/engine/installation/linux/ubuntulinux/>`__ for
installation instructions. Please always install the **latest version** (currently version 12) and do not use
outdated packages from your Linux distribution. Docker provides third-party repositories for all major platforms.

CC-Sample-App
-------------

The CC-Sample-App is a containerized application, which is compatible with CC-Server because it is based on
CC-Image-Ubuntu. There are several CC-Image flavors, that can be used as a base image for application containers.
They contain the Python code of CC-Container-Worker.

The source code of the images can be found on Github. All images have been uploaded to Docker Hub. The follwoing table
contains the corresponding web links and how to refer to the image when using docker-engine or a Dockerfile.

======================  =========================================================================  ===================================================================================  ==================================================
Image                   Code                                                                       Registry                                                                             Docker URL
======================  =========================================================================  ===================================================================================  ==================================================
CC-Sample-App           `Github <https://github.com/curious-containers/cc-sample-app>`__           `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-sample-app/>`__           docker.io/curiouscontainers/cc-sample-app
CC-Image-Ubuntu         `Github <https://github.com/curious-containers/cc-image-ubuntu>`__         `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-image-ubuntu/>`__         docker.io/curiouscontainers/cc-image-ubuntu
CC-Image-Fedora         `Github <https://github.com/curious-containers/cc-image-fedora>`__         `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-image-fedora/>`__         docker.io/curiouscontainers/cc-image-fedora
CC-Image-Debian         `Github <https://github.com/curious-containers/cc-image-debian>`__         `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-image-debian/>`__         docker.io/curiouscontainers/cc-image-debian
======================  =========================================================================  ===================================================================================  ==================================================

The CC-Sample-App contains the bash script *algorithm.sh* which is a minimal program that can be executed by
CC-Container-Worker. As can be seen in the source code of *algorithm.sh* below, the script does two different things.
It first copies the file *data.txt* from the *input_files* directory to the *result_files* directory.
The second command takes the CLI arguments *${@}* and writes them to the *parameters.txt* file. Please note that the
*data.txt* file will only be copied with a chance of 50%, making it an **optional** result file.

.. code-block:: bash

   if [[ $(($RANDOM % 2)) == 0 ]]; then
       cp /home/ubuntu/input_files/data.txt /home/ubuntu/result_files/data.txt
   fi
   echo ${@} > /home/ubuntu/result_files/parameters.txt


The algorithm assumes to find one input file at a specific location in the local file system of the container image.
In addition both result files are written to the file system at specific locations. Since the *algorithm.sh* script will
be executed by CC-Container-Worker it is necessary to inform the worker about the locations of the
input file and the result files by creating a *config.json* file as shown below. The field **application_command** defines
that the script will be executed with *bash* and that it is located in the system users home directory.
The **local_input_files** field contains a list with dictionaries describing the file locations.
The **local_result_files** contains a dictionary, where the keys in the dictionary are identifiers for the files, which
can be chosen freely. The value for each key is a dictionary describing the file location. The worker software will
throw an error, if a specified result file is not created by the application. Result files can be marked as **optional**,
in order to avoid this behaviour.

.. code-block:: json

   {
      "application_command": "bash /home/ubuntu/algorithm.sh",
      "local_input_files": [
          {"dir": "/home/ubuntu/input_files", "name": "data.txt"}
      ],
      local_result_files = {
          "file_a": {"dir": "/home/ubuntu/result_files", "name": "data.txt", "optional": true},
          "file_b": {"dir": "/home/ubuntu/result_files", "name": "parameters.txt"}
      }


In order to create the CC-Sample-App it is necessary to build a Docker image, containing *algorithm.sh* and *config.toml*.
For this purpose a Dockerfile must be defined as follows. The **FROM** statement defines the base image, in this case
CC-Image-Ubuntu was chosen. The first **COPY** statement copies *config.toml* to the */opt* directory in the container images
file system. This path cannot be changed, because CC-Container-Worker assumes the file to be at this specific location.
The second **COPY** statement copies the application *algorithm.sh* to the location previously defined in *config.toml*
via **application_command**.

.. code-block:: docker

   FROM docker.io/curiouscontainers/cc-image-ubuntu:0.7<commit_msg>small fix in user docs<commit_after>User Documentation
==================

This user documentation contains information about how to schedule a task with CC-Server to run the CC-Sample-App and
how to build a custom Docker images containing arbitrary applications.

Docker Installation
-------------------

Please follow the official `Docker documenation <https://docs.docker.com/engine/installation/linux/ubuntulinux/>`__ for
installation instructions. Please always install the **latest version** (currently version 12) and do not use
outdated packages from your Linux distribution. Docker provides third-party repositories for all major platforms.

CC-Sample-App
-------------

The CC-Sample-App is a containerized application, which is compatible with CC-Server because it is based on
CC-Image-Ubuntu. There are several CC-Image flavors, that can be used as a base image for application containers.
They contain the Python code of CC-Container-Worker.

The source code of the images can be found on Github. All images have been uploaded to Docker Hub. The follwoing table
contains the corresponding web links and how to refer to the image when using docker-engine or a Dockerfile.

======================  =========================================================================  ===================================================================================  ==================================================
Image                   Code                                                                       Registry                                                                             Docker URL
======================  =========================================================================  ===================================================================================  ==================================================
CC-Sample-App           `Github <https://github.com/curious-containers/cc-sample-app>`__           `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-sample-app/>`__           docker.io/curiouscontainers/cc-sample-app
CC-Image-Ubuntu         `Github <https://github.com/curious-containers/cc-image-ubuntu>`__         `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-image-ubuntu/>`__         docker.io/curiouscontainers/cc-image-ubuntu
CC-Image-Fedora         `Github <https://github.com/curious-containers/cc-image-fedora>`__         `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-image-fedora/>`__         docker.io/curiouscontainers/cc-image-fedora
CC-Image-Debian         `Github <https://github.com/curious-containers/cc-image-debian>`__         `Docker Hub <https://hub.docker.com/r/curiouscontainers/cc-image-debian/>`__         docker.io/curiouscontainers/cc-image-debian
======================  =========================================================================  ===================================================================================  ==================================================

The CC-Sample-App contains the bash script *algorithm.sh* which is a minimal program that can be executed by
CC-Container-Worker. As can be seen in the source code of *algorithm.sh* below, the script does two different things.
It first copies the file *data.txt* from the *input_files* directory to the *result_files* directory.
The second command takes the CLI arguments *${@}* and writes them to the *parameters.txt* file. Please note that the
*data.txt* file will only be copied with a chance of 50%, making it an **optional** result file.

.. code-block:: bash

   if [[ $(($RANDOM % 2)) == 0 ]]; then
       cp /home/ubuntu/input_files/data.txt /home/ubuntu/result_files/data.txt
   fi
   echo ${@} > /home/ubuntu/result_files/parameters.txt


The algorithm assumes to find one input file at a specific location in the local file system of the container image.
In addition both result files are written to the file system at specific locations. Since the *algorithm.sh* script will
be executed by CC-Container-Worker it is necessary to inform the worker about the locations of the
input file and the result files by creating a *config.json* file as shown below. The field **application_command** defines
that the script will be executed with *bash* and that it is located in the system users home directory.
The **local_input_files** field contains a list with dictionaries describing the file locations.
The **local_result_files** contains a dictionary, where the keys in the dictionary are identifiers for the files, which
can be chosen freely. The value for each key is a dictionary describing the file location. The worker software will
throw an error, if a specified result file is not created by the application. Result files can be marked as **optional**,
in order to avoid this behaviour.

.. code-block:: json

   {
      "application_command": "bash /home/ubuntu/algorithm.sh",
      "local_input_files": [
          {"dir": "/home/ubuntu/input_files", "name": "data.txt"}
      ],
      local_result_files = {
          "file_a": {"dir": "/home/ubuntu/result_files", "name": "data.txt", "optional": true},
          "file_b": {"dir": "/home/ubuntu/result_files", "name": "parameters.txt"}
      }
   }


In order to create the CC-Sample-App it is necessary to build a Docker image, containing *algorithm.sh* and *config.toml*.
For this purpose a Dockerfile must be defined as follows. The **FROM** statement defines the base image, in this case
CC-Image-Ubuntu was chosen. The first **COPY** statement copies *config.toml* to the */opt* directory in the container images
file system. This path cannot be changed, because CC-Container-Worker assumes the file to be at this specific location.
The second **COPY** statement copies the application *algorithm.sh* to the location previously defined in *config.toml*
via **application_command**.

.. code-block:: docker

   FROM docker.io/curiouscontainers/cc-image-ubuntu:0.7
{'fraction_non_alphanumeric': 0.20109704641350212, 'fraction_numerical': 0.001350210970464135, 'mean_word_length': 3.807707910750507, 'pattern_counts': {'":': 22, '<': 21, '<?xml version=': 0, '>': 23, 'https://': 18, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2585695', 'n_tokens_mistral': 2989, 'n_tokens_neox': 2729, 'n_words': 1137}