<commit_before>.. _administrator:

Administrator Manual
####################

OpenSubmit consists of two parts: The web application and the executor daemon. The web application provides the user interface for students and teachers, while the executor daemons evaluate student code with according test scripts.

.. image:: files/architecture.png    

If you just want to play around, use our `demo installation <http://www.demo.open-submit.org>`_.

If you want your own production setup, go the manual way for :ref:`web application <manualweb>` and :ref:`test machines <manualexec>`.

Please note that OpenSubmit :ref:`does not support password-based login <principles>`. You need to work with one of the supported :ref:`auth`.

.. _Terraform:

Full-stack installation with Terraform
**************************************

The source code repository offers a  `Terraform <http://terraform.io>`_ script for deploying a complete OpenSubmit environment on a single machine running in the `Google Compute Engine <https://cloud.google.com/compute>`_. The installation procedure deploys the Docker containers described in the following sections. For such an installation:

- Install `Terraform <http://terraform.io>`_ on your local machine.
- Clone the Git repository for OpenSubmit and adjust the variables in `terraform.tf <https://github.com/troeger/opensubmit/blob/master/terraform.tf>`_.
- Call ``terraform apply``.

This setup is not recommended for production, since the database is installed as Docker image. 

Full-stack installation with Docker Compose
*******************************************

You can replicate our `demo installation <http://www.demo.open-submit.org>`_ on your own machine with `Docker Compose <https://docs.docker.com/compose/overview/>`_, which comes as part of a normal `Docker installation <https://www.docker.com/community-edition#/download>`_. Our compose file relies on the official Docker images for the `web application <https://hub.docker.com/r/troeger/opensubmit-web/>`_ and the `executor <https://hub.docker.com/r/troeger/opensubmit-exec/>`_.

- Download the `compose file <https://raw.githubusercontent.com/troeger/opensubmit/master/docker-compose.yml>`_ on the machine.
- Call ``docker-compose up`` to download, configure and start the OpenSubmit Docker containers and a separate database container.
- Got to ``http://localhost:8000`` and use one of the configured authentication methods.

This setup is not recommended for production, since the database is installed as Docker image. 

Single installation of the web application
******************************************

The OpenSubmit web application runs with Python 3.4 or newer versions. There are two options:

.. _dockerweb:

Docker-based installation
=========================

The latest official release of the OpenSubmit web application is available as single `opensubmit-web Docker image <https://hub.docker.com/r/troeger/opensubmit-web/>`_. It expects a couple of environment variables to be set, check the :ref:`configuration section <config_web>` for further details.

.. _manualweb:

Manual installation
===================

This is the recommended approach for production environments. You need to follow these steps:
  
- Prepare a Python 3 web hosting environment. 

  - Debian / Ubuntu: ``apt-get install libapache2-mod-wsgi-py3 apache2 sqlite python3``. 

- Run ``pip install opensubmit-web`` as root or in a virtualenv environment. If you get error messages about unresolved dependencies, try running ``pip install -U opensubmit-web``. PIP should come as part of your Python installation.
- Run ``opensubmit-web configcreate`` to create an OpenSubmit configuration file. Check the :ref:`configuration section <config_web>` for details.  
- Run ``opensubmit-web configtest`` to check your configuration.
- Run ``opensubmit-web apachecreate`` to generate a default Apache 2.4 configuration for mod_wsgi, which is stored in ``/etc/opensubmit/apache24.config``.  You can `include <http://httpd.apache.org/docs/2.4/en/mod/core.html#include>`_ this file in some `virtual host configuration <http://httpd.apache.org/docs/2.4/vhosts/examples.html>`_.
- Restart your web server.
- Got to the OpenSubmit start page and use your configured authentication method.
- Run ``opensubmit-web makeadmin <email>`` to make the created user an administrator in the system.

Updating an existing manual installation is easy:

- Run ``pip install --upgrade opensubmit-web`` as root or in the virtualenv environment. 
- Run ``opensubmit-web configtest`` to perform neccessary database updates.
- Restart your web server.


.. _config_web:

Configuration of the web application
====================================

OpenSubmit searches for a configuration file in ``/etc/opensubmit/settings.ini``. This file should be initially created by calling ``opensubmit-web configcreate``. This management command allows to pre-define specific configuration options via command-line or environment variables, and creates an according config file. Check ``opensubmit-web configcreate -h`` for details.

Impress and privacy policy
--------------------------

There are several European regulations that expect a web page to provide both an impress and a privacy policy page (GDPR / DSGVO). There are two ways to achieve that:

- Option 1: Your configuration file defines name, address, and email of an administrator. The according options for ``opensubmit-web configcreate`` are ``--admin_name``, ``--admin_email``, and ``--admin_address``. If you want to modify settings.ini directly, add ``ADMIN_NAME``, ``ADMIN_EMAIL`` and ``ADMIN_ADDRESS`` in the ``[admin]`` section. The first two settings are mandatory anyway. Given that information, OpenSubmit will provide a default impress and privacy policy page.
<commit_msg>Add explicit configuration docs<commit_after>.. _administrator:

Administrator Manual
####################

OpenSubmit consists of two parts: The web application and the executor daemon. The web application provides the user interface for students and teachers, while the executor daemons evaluate student code with according test scripts.

.. image:: files/architecture.png    

If you just want to play around, use our `demo installation <http://www.demo.open-submit.org>`_.

If you want your own production setup, go the manual way for :ref:`web application <manualweb>` and :ref:`test machines <manualexec>`.

Please note that OpenSubmit :ref:`does not support password-based login <principles>`. You need to work with one of the supported :ref:`auth`.

.. _Terraform:

Full-stack installation with Terraform
**************************************

The source code repository offers a  `Terraform <http://terraform.io>`_ script for deploying a complete OpenSubmit environment on a single machine running in the `Google Compute Engine <https://cloud.google.com/compute>`_. The installation procedure deploys the Docker containers described in the following sections. For such an installation:

- Install `Terraform <http://terraform.io>`_ on your local machine.
- Clone the Git repository for OpenSubmit and adjust the variables in `terraform.tf <https://github.com/troeger/opensubmit/blob/master/terraform.tf>`_.
- Call ``terraform apply``.

This setup is not recommended for production, since the database is installed as Docker image. 

Full-stack installation with Docker Compose
*******************************************

You can replicate our `demo installation <http://www.demo.open-submit.org>`_ on your own machine with `Docker Compose <https://docs.docker.com/compose/overview/>`_, which comes as part of a normal `Docker installation <https://www.docker.com/community-edition#/download>`_. Our compose file relies on the official Docker images for the `web application <https://hub.docker.com/r/troeger/opensubmit-web/>`_ and the `executor <https://hub.docker.com/r/troeger/opensubmit-exec/>`_.

- Download the `compose file <https://raw.githubusercontent.com/troeger/opensubmit/master/docker-compose.yml>`_ on the machine.
- Call ``docker-compose up`` to download, configure and start the OpenSubmit Docker containers and a separate database container.
- Got to ``http://localhost:8000`` and use one of the configured authentication methods.

This setup is not recommended for production, since the database is installed as Docker image. 

Single installation of the web application
******************************************

The OpenSubmit web application runs with Python 3.4 or newer versions. There are two options:

.. _dockerweb:

Docker-based installation
=========================

The latest official release of the OpenSubmit web application is available as single `opensubmit-web Docker image <https://hub.docker.com/r/troeger/opensubmit-web/>`_. It expects a couple of environment variables to be set, check the :ref:`configuration section <config_web>` for further details.

.. _manualweb:

Manual installation
===================

This is the recommended approach for production environments. You need to follow these steps:
  
- Prepare a Python 3 web hosting environment. 

  - Debian / Ubuntu: ``apt-get install libapache2-mod-wsgi-py3 apache2 sqlite python3``. 

- Run ``pip install opensubmit-web`` as root or in a virtualenv environment. If you get error messages about unresolved dependencies, try running ``pip install -U opensubmit-web``. PIP should come as part of your Python installation.
- Run ``opensubmit-web configcreate`` to create an OpenSubmit configuration file. Check the :ref:`configuration section <config_web>` for details.  
- Run ``opensubmit-web configtest`` to check your configuration.
- Run ``opensubmit-web apachecreate`` to generate a default Apache 2.4 configuration for mod_wsgi, which is stored in ``/etc/opensubmit/apache24.config``.  You can `include <http://httpd.apache.org/docs/2.4/en/mod/core.html#include>`_ this file in some `virtual host configuration <http://httpd.apache.org/docs/2.4/vhosts/examples.html>`_.
- Restart your web server.
- Got to the OpenSubmit start page and use your configured authentication method.
- Run ``opensubmit-web makeadmin <email>`` to make the created user an administrator in the system.

Updating an existing manual installation is easy:

- Run ``pip install --upgrade opensubmit-web`` as root or in the virtualenv environment. 
- Run ``opensubmit-web configtest`` to perform neccessary database updates.
- Restart your web server.


.. _config_web:

Configuration of the web application
************************************

OpenSubmit searches for a configuration file in ``/etc/opensubmit/settings.ini``. This file should be initially created by calling ``opensubmit-web configcreate``. The command allows to pre-define specific configuration options via command-line, or environment variables, and creates an according config file. 

=============================== ===================================== ============================================================================
Command-line option             Environment variable                  Description
=============================== ===================================== ============================================================================
--debug                         OPENSUBMIT_DEBUG                      Enable debug mode, not for production systems
--server_url                    OPENSUBMIT_SERVER_URL                 The main URL of the OpenSubmit installation, including sub-directories
--server_mediaroot              OPENSUBMIT_SERVER_MEDIAROOT           Storage path for uploadeded files
--server_hostaliases            OPENSUBMIT_SERVER_HOSTALIASES         Comma-separated list of alternative host names for the web server
--server_logfile                OPENSUBMIT_SERVER_LOGFILE             Log file for the OpenSubmit application
--server_timezone               OPENSUBMIT_SERVER_TIMEZONE            Time zone for all dates and deadlines
--database_name                 OPENSUBMIT_DATABASE_NAME              Name of the database (file)
--database_engine               OPENSUBMIT_DATABASE_ENGINE            Datababase engine being used
--database_user                 OPENSUBMIT_DATABASE_USER              The user name for accessing the database. Not needed for SQLite
--database_password             OPENSUBMIT_DATABASE_PASSWORD          The user password for accessing the database. Not needed for SQLite
--database_host                 OPENSUBMIT_DATABASE_HOST              The host name for accessing the database. Not needed for SQLite
--database_port                 OPENSUBMIT_DATABASE_PORT              The port number for accessing the database. Not needed for SQLite
--login_google_oauth_key        OPENSUBMIT_LOGIN_GOOGLE_OAUTH_KEY     Google OAuth client key
--login_google_oauth_secret     OPENSUBMIT_LOGIN_GOOGLE_OAUTH_SECRET  Google OAuth client secret
--login_twitter_oauth_key       OPENSUBMIT_LOGIN_TWITTER_OAUTH_KEY    Twitter OAuth client key
--login_twitter_oauth_secret    OPENSUBMIT_LOGIN_TWITTER_OAUTH_SECRET Twitter OAuth client secret
--login_github_oauth_key        OPENSUBMIT_LOGIN_GITHUB_OAUTH_KEY     GitHub OAuth client key
--login_github_oauth_secret     OPENSUBMIT_LOGIN_GITHUB_OAUTH_SECRET  GitHub OAuth client secret
--login_openid_description      OPENSUBMIT_LOGIN_OPENID_DESCRIPTION   Title of the OpenID login button
--login_openid_provider         OPENSUBMIT_LOGIN_OPENID_PROVIDER      URL of the OpenID provider
--login_oidc_description        OPENSUBMIT_LOGIN_OIDC_DESCRIPTION     Title of the OpenID Connect login button
--login_oidc_endpoint           OPENSUBMIT_LOGIN_OIDC_ENDPOINT        URL of the OpenID Connect endpoint
--login_oidc_client_id          OPENSUBMIT_LOGIN_OIDC_CLIENT_ID       OpenID Connect client id
--login_oidc_client_secret      OPENSUBMIT_LOGIN_OIDC_CLIENT_SECRET   OpenID Connect client secret
--login_shib_description        OPENSUBMIT_LOGIN_SHIB_DESCRIPTION     Title of the Shibboleth login button
--login_demo                    OPENSUBMIT_LOGIN_DEMO                 Offer demo login options
--admin_name                    OPENSUBMIT_ADMIN_NAME                 Name of the administrator, shown in privacy policy, impress and backend
--admin_email                   OPENSUBMIT_ADMIN_EMAIL                eMail of the administrator, shown in privacy policy, impress and backend
--admin_address                 OPENSUBMIT_ADMIN_ADDRESS              Address of the administrator, shown in privacy policy and impress
--admin_impress_page            OPENSUBMIT_IMPRESS_PAGE               Link to alternative impress page
--admin_privacy_page            OPENSUBMIT_PRIVACY_PAGE               Link to alternative privacy policy page
=============================== ===================================== ============================================================================

Check ``opensubmit-web configcreate -h`` for details.

The environment variables can also be used to configure the Docker version of OpenSubmit.

Impress and privacy policy
==========================

There are several European regulations that expect a web page to provide both an impress and a privacy policy page (GDPR / DSGVO). There are two ways to achieve that:

- Option 1: Your configuration file defines name, address, and email of an administrator. The according options for ``opensubmit-web configcreate`` are ``--admin_name``, ``--admin_email``, and ``--admin_address``. If you want to modify settings.ini directly, add ``ADMIN_NAME``, ``ADMIN_EMAIL`` and ``ADMIN_ADDRESS`` in the ``[admin]`` section. The first two settings are mandatory anyway. Given that information, OpenSubmit will provide a default impress and privacy policy page.

{'fraction_non_alphanumeric': 0.1255336774357994, 'fraction_numerical': 0.0025489071560568406, 'mean_word_length': 4.191531591134634, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 16, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4641925', 'n_tokens_mistral': 4092, 'n_tokens_neox': 3831, 'n_words': 1601}