<commit_before>WMarket: Installation and Administration Guide
==============================================

The purpose of this document is to describe how to install and
administrate the necessary software on a server so that it can run
WMarket.

WMarket itself is a Java Web Application, packaged in a WAR file and
relys on a SQL database.

System Requirements
===================

This section covers the requirements needed to install and use WMarket.

Hardware Requirements
---------------------

The following table contains the minimum resource requirements for
running WMarket:

+----------------+--------------------------------------------------------------------------------------------+
| Resource       | Requirement                                                                                |
+================+============================================================================================+
| CPU            | 1-2 cores with at leas 2.4 GHz                                                             |
+----------------+--------------------------------------------------------------------------------------------+
| Physical RAM   | 2GB-4GB                                                                                    |
+----------------+--------------------------------------------------------------------------------------------+
| Disk Space     | 10GB The actual disk space will depend on the amount of data being stored within WMarket   |
+----------------+--------------------------------------------------------------------------------------------+

Operating System Support
------------------------

WMarket has been tested against the following Operating Systems: 

- Ubuntu 14.04 LTS 
- CentOS 7

However, you can install WMarket in any machine where you can install an
Application Server (such as Tomcat) and a MySQL server.

.. note::

  This Installation Guide describes the isntalltion process on a
  Linux based System.

Software Requirements
---------------------

In order to have WMarket running, the following software is needed:

-  MySQL Server 5.5
-  Java 1.7.x
-  Application Server, Apache Tomcat 7.x
-  WMarket Software

Software Installation
---------------------

Installing WMarket using scripts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to ease the WMarket installation, the script ``install.sh`` is
provided. This script installs all the needed dependencies, configures
the WMarket instance (database and preferences) and deploys it in the
Tomcat Server. To use this script execute the command:

::

    $ sudo ./install.sh

.. note::
  If you use this script you can skip the configuration section;
  however, it is highly recommended to read it in order to understand the
  different configuration parameters.

During the execution of the script you will be prompted several times in
order to retrieve some information. Specifically:

-  **Database Installation**: in Debian/Ubuntu systems the MySQL
   installer automatically ask a password for the creation of the root
   user. In CentOS systems the script calls the MySQL configuration
   command in order to create this password so a message similar to the
   following one will be shown:

::

    NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
          SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

    In order to log into MySQL to secure it, we'll need the current
    password for the root user.  If you've just installed MySQL, and
    you haven't set the root password yet, the password will be blank,
    so you should just press enter here.

    Enter current password for root (enter for none): 
    OK, successfully used password, moving on...

    Setting the root password ensures that nobody can log into the MySQL
    root user without the proper authorisation.

    Set root password? [Y/n] y  
    New password: *******
    Re-enter new password: *******
    Password updated successfully!
    Reloading privilege tables..
     ... Success!


    By default, a MySQL installation has an anonymous user, allowing anyone
    to log into MySQL without having to have a user account created for
    them.  This is intended only for testing, and to make the installation
    go a bit smoother.  You should remove them before moving into a
    production environment.

    Remove anonymous users? [Y/n] y
     ... Success!

    Normally, root should only be allowed to connect from 'localhost'.  This
    ensures that someone cannot guess at the root password from the network.

    Disallow root login remotely? [Y/n] y
     ... Success!

    By default, MySQL comes with a database named 'test' that anyone can
    access.  This is also intended only for testing, and should be removed
    before moving into a production environment.

    Remove test database and access to it? [Y/n] y
     - Dropping test database...
    ERROR 1008 (HY000) at line 1: Can't drop database 'test'; database doesn't exist
     ... Failed!  Not critical, keep moving...
     - Removing privileges on test database...
     ... Success!

    Reloading the privilege tables will ensure that all changes made so far
    will take effect immediately.

    Reload privilege tables now? [Y/n] y
     ... Success!




    All done!  If you've completed all of the above steps, your MySQL
    installation should now be secure.

    Thanks for using MySQL!


    Cleaning up...

-  **Database Configuration**: the script creates a database called
   ``marketplace``. To perfom this action, the script needs MySQL
   credentials with administrative permission. For this reason, you will
   see the following lines:

::

    > About to create 'marketplace' database. Please, provide MySQL credentials with administrative rights (i.e. root user)
    >> User: root
    >> Password: *******

-  **Authentication Configuration**: WMarket can work with local
   authentication or using an external IdM. The script will ask you what
   authentication system do you prefer. If you opt for an external IdM,
   some configuration details will be required as can be seen in the
   following example:

::

    > Do you want to use OAuth2 to authenticate users? (Y/n): Y
    >> OAuth2 Server: [FIWARE_IDM_URL]
    >> OAuth2 Key: [OAUTH2_KEY]
    >> OAuth2 Secret: [OAUTH2_SECRET]
    >> Provider Role: [OAUTH2_PROVIDER_ROLE]
    >> WMarket External IP: [MACHINE_IP]

This script will directly configure some preferences with default values
(see the *Configuration* section for more info of the described
preferences). Concretely:

-  A database called ``marketplace`` is created.
-  Database connection is configured with the administrative credentials
   you have provided.
-  Authentication system is configured according to your preferences.
-  The Marketplace is deployed in Tomcat as ``WMarket``.

Manually installing WMarket
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Ubuntu/Debian
^^^^^^^^^^^^^

All the mandatory dependencies can be easily installed on a debian based
Linux distribution using ``apt-get``:

::

    sudo apt-get install mysql-server mysql-client
    sudo apt-get install openjdk-7-jdk
    sudo apt-get install tomcat7 tomcat7-docs tomcat7-admin

CentOS/RedHat
^^^^^^^^^^^^^

In a CentOS/RedHat system, all the required dependencies can be
installed using ``yum``:

::

    # Install MySQL
    sudo rpm -Uvh http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
    sudo yum -y install mysql-community-server

    # Install Java
    sudo yum -y install java-1.7.0-openjdk-devel

    # Install Tomcat 7
    sudo yum -y install tomcat tomcat-webapps tomcat-admin-webapps

Configuration
-------------

This section explains how to create WMarket database and how to
configure the different preferences. Note that if you have used the
provided script you can just skip this section. However, it is highly
recommended to read it in order to understand the different settings.

Database Configuration
~~~~~~~~~~~~~~~~~~~~~~

WMarket uses an internal database named ``marketplace`` that need to be
created in MySQL. To create it you need to have administrator
permissions in MySQL. This usually means that you have to use the MySQL
root user with the password you chose during the installation process.

Ubuntu/Debian:
^^^^^^^^^^^^^^

::

    sudo service mysql start
    mysqladmin -u root -p[MYSQL_ROOT_PWD] create marketplace

CentOS/RedHat:
^^^^^^^^^^^^^^

::

    sudo systemctl start mysqld
    mysqladmin -u root -p[MYSQL_ROOT_PWD] create marketplace

WMarket Configuration
~~~~~~~~~~~~~~~~~~~~~

Before deploying the provider JAR into your Application Server, you must
configure some parameters.

Database
^^^^^^^^

Before you deploy WMarket, you have to set up the database. To do so,
you have to edit the file
``WMarket.war/WEB-INF/classes/properties/database.properties`` and
complete the fields as follows:

::

    jdbc.driverClassName=com.mysql.jdbc.Driver
    jdbc.url=jdbc:mysql://[MYSQL_SERVER]:[MYSQL_PORT]/marketplace
    jdbc.username=[YOUR_DB_USER]
    jdbc.password=[<PASSWORD>]

Index
^^^^^

WMarket uses a index files to provide better search results. These files
must me stored in some folder of your file system. You can specify this
folder by editing the property ``lucene.IndexPath`` included in the file
``WMarket.war/WEB-INF/classes/properties/marketplace.properties``.

OAuth2
^^^^^^

WMarket uses a local authentication system by default. However, the
software is ready to work with the FIWARE Identity Manager.

The first thing that you have to do is to create a new application in
the FIWARE IdM. To do so, go to
https://account.lab.fiware.org/idm/myApplications/create/ and complete
all the required fields:

-  **Name**: *You can choose any name*
-  **Description**: *You can write any description*
-  **URL**: ``http://[WMARKET_HOST]:[WMARKET_PORT]``
-  **Callback URL**:
   ``http://[WMARKET_HOST]:[WMARKET_PORT]/WMarket/callback?client_name=FIWAREClient``
-  **Roles**: In order to allow some users to create stores and
   descriptions with the user interface, you must create a new role. You
   can choose any name.

Once that you have created the application in the FIWARE IdM, you must
edit the following configuration files:

1. ``WMarket.war/WEB-INF/classes/spring/config/BeanLocations.xml``:<commit_msg>Fix typos<commit_after>WMarket: Installation and Administration Guide
==============================================

The purpose of this document is to describe how to install and
administrate the necessary software on a server so that it can run
WMarket.

WMarket itself is a Java Web Application, packaged in a WAR file and
relys on a SQL database.

System Requirements
===================

This section covers the requirements needed to install and use WMarket.

Hardware Requirements
---------------------

The following table contains the minimum resource requirements for
running WMarket:

+----------------+--------------------------------------------------------------------------------------------+
| Resource       | Requirement                                                                                |
+================+============================================================================================+
| CPU            | 1-2 cores with at leas 2.4 GHz                                                             |
+----------------+--------------------------------------------------------------------------------------------+
| Physical RAM   | 2GB-4GB                                                                                    |
+----------------+--------------------------------------------------------------------------------------------+
| Disk Space     | 10GB The actual disk space will depend on the amount of data being stored within WMarket   |
+----------------+--------------------------------------------------------------------------------------------+

Operating System Support
------------------------

WMarket has been tested against the following Operating Systems: 

- Ubuntu 14.04 LTS 
- CentOS 7

However, you can install WMarket in any machine where you can install an
Application Server (such as Tomcat) and a MySQL server.

.. note::

  This Installation Guide describes the installation process on a
  Linux based System.

Software Requirements
---------------------

In order to have WMarket running, the following software is needed:

-  MySQL Server 5.5
-  Java 1.7.x
-  Application Server, Apache Tomcat 7.x
-  WMarket itself

Software Installation
---------------------

Installing WMarket using scripts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to ease the WMarket installation, the script ``install.sh`` is
provided. This script installs all the needed dependencies, configures
the WMarket instance (database and preferences) and deploys it in the
Tomcat Server. To use this script execute the command:

::

    $ sudo ./install.sh

.. note::
  If you use this script you can skip the configuration section;
  however, it is highly recommended to read it in order to understand the
  different configuration parameters.

During the execution of the script you will be prompted several times in
order to retrieve some information. Specifically:

-  **Database Installation**: in Debian/Ubuntu systems the MySQL
   installer automatically ask a password for the creation of the root
   user. In CentOS systems the script calls the MySQL configuration
   command in order to create this password so a message similar to the
   following one will be shown:

::

    NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
          SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

    In order to log into MySQL to secure it, we'll need the current
    password for the root user.  If you've just installed MySQL, and
    you haven't set the root password yet, the password will be blank,
    so you should just press enter here.

    Enter current password for root (enter for none): 
    OK, successfully used password, moving on...

    Setting the root password ensures that nobody can log into the MySQL
    root user without the proper authorisation.

    Set root password? [Y/n] y  
    New password: *******
    Re-enter new password: *******
    Password updated successfully!
    Reloading privilege tables..
     ... Success!


    By default, a MySQL installation has an anonymous user, allowing anyone
    to log into MySQL without having to have a user account created for
    them.  This is intended only for testing, and to make the installation
    go a bit smoother.  You should remove them before moving into a
    production environment.

    Remove anonymous users? [Y/n] y
     ... Success!

    Normally, root should only be allowed to connect from 'localhost'.  This
    ensures that someone cannot guess at the root password from the network.

    Disallow root login remotely? [Y/n] y
     ... Success!

    By default, MySQL comes with a database named 'test' that anyone can
    access.  This is also intended only for testing, and should be removed
    before moving into a production environment.

    Remove test database and access to it? [Y/n] y
     - Dropping test database...
    ERROR 1008 (HY000) at line 1: Can't drop database 'test'; database doesn't exist
     ... Failed!  Not critical, keep moving...
     - Removing privileges on test database...
     ... Success!

    Reloading the privilege tables will ensure that all changes made so far
    will take effect immediately.

    Reload privilege tables now? [Y/n] y
     ... Success!




    All done!  If you've completed all of the above steps, your MySQL
    installation should now be secure.

    Thanks for using MySQL!


    Cleaning up...

-  **Database Configuration**: the script creates a database called
   ``marketplace``. To perfom this action, the script needs MySQL
   credentials with administrative permission. For this reason, you will
   see the following lines:

::

    > About to create 'marketplace' database. Please, provide MySQL credentials with administrative rights (i.e. root user)
    >> User: root
    >> Password: *******

-  **Authentication Configuration**: WMarket can work with local
   authentication or using an external IdM. The script will ask you what
   authentication system do you prefer. If you opt for an external IdM,
   some configuration details will be required as can be seen in the
   following example:

::

    > Do you want to use OAuth2 to authenticate users? (Y/n): Y
    >> OAuth2 Server: [FIWARE_IDM_URL]
    >> OAuth2 Key: [OAUTH2_KEY]
    >> OAuth2 Secret: [OAUTH2_SECRET]
    >> Provider Role: [OAUTH2_PROVIDER_ROLE]
    >> WMarket External IP: [MACHINE_IP]

This script will directly configure some preferences with default values
(see the *Configuration* section for more info of the described
preferences). Concretely:

-  A database called ``marketplace`` is created.
-  Database connection is configured with the administrative credentials
   you have provided.
-  Authentication system is configured according to your preferences.
-  The Marketplace is deployed in Tomcat as ``WMarket``.

Manually installing WMarket
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Ubuntu/Debian
^^^^^^^^^^^^^

All the mandatory dependencies can be easily installed on a Debian based
Linux distribution using ``apt-get``:

::

    sudo apt-get install mysql-server mysql-client
    sudo apt-get install openjdk-7-jdk
    sudo apt-get install tomcat7 tomcat7-docs tomcat7-admin

CentOS/RedHat
^^^^^^^^^^^^^

In a CentOS/RedHat system, all the required dependencies can be
installed using ``yum``:

::

    # Install MySQL
    sudo rpm -Uvh http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
    sudo yum -y install mysql-community-server

    # Install Java
    sudo yum -y install java-1.7.0-openjdk-devel

    # Install Tomcat 7
    sudo yum -y install tomcat tomcat-webapps tomcat-admin-webapps

Configuration
-------------

This section explains how to create WMarket database and how to
configure the different preferences. Note that if you have used the
provided script you can just skip this section. However, it is highly
recommended to read it in order to understand the different settings.

Database Configuration
~~~~~~~~~~~~~~~~~~~~~~

WMarket uses an internal database named ``marketplace`` that need to be
created in MySQL. To create it you need to have administrator
permissions in MySQL. This usually means that you have to use the MySQL
root user with the password you chose during the installation process.

Ubuntu/Debian:
^^^^^^^^^^^^^^

::

    sudo service mysql start
    mysqladmin -u root -p[MYSQL_ROOT_PWD] create marketplace

CentOS/RedHat:
^^^^^^^^^^^^^^

::

    sudo systemctl start mysqld
    mysqladmin -u root -p[MYSQL_ROOT_PWD] create marketplace

WMarket Configuration
~~~~~~~~~~~~~~~~~~~~~

Before deploying the provider JAR into your Application Server, you must
configure some parameters.

Database
^^^^^^^^

Before you deploy WMarket, you have to set up the database. To do so,
you have to edit the file
``WMarket.war/WEB-INF/classes/properties/database.properties`` and
complete the fields as follows:

::

    jdbc.driverClassName=com.mysql.jdbc.Driver
    jdbc.url=jdbc:mysql://[MYSQL_SERVER]:[MYSQL_PORT]/marketplace
    jdbc.username=[YOUR_DB_USER]
    jdbc.password=[<PASSWORD>]

Index
^^^^^

WMarket uses index files to provide better search results. These files
must me stored in some folder of your file system. You can specify this
folder by editing the property ``lucene.IndexPath`` included in the file
``WMarket.war/WEB-INF/classes/properties/marketplace.properties``.

OAuth2
^^^^^^

WMarket uses a local authentication system by default. However, the
software is ready to work with the FIWARE Identity Manager.

The first thing that you have to do is to create a new application in
the FIWARE IdM. To do so, go to
https://account.lab.fiware.org/idm/myApplications/create/ and complete
all the required fields:

-  **Name**: *You can choose any name*
-  **Description**: *You can write any description*
-  **URL**: ``http://[WMARKET_HOST]:[WMARKET_PORT]``
-  **Callback URL**:
   ``http://[WMARKET_HOST]:[WMARKET_PORT]/WMarket/callback?client_name=FIWAREClient``
-  **Roles**: In order to allow some users to create stores and
   descriptions with the user interface, you must create a new role. You
   can choose any name for this role.

Once that you have created the application in the FIWARE IdM, you must
edit the following configuration files:

1. ``WMarket.war/WEB-INF/classes/spring/config/BeanLocations.xml``:
{'fraction_non_alphanumeric': 0.14200436787187576, 'fraction_numerical': 0.004367871875758311, 'mean_word_length': 2.720837847598411, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 37, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4301862', 'n_tokens_mistral': 5327, 'n_tokens_neox': 4929, 'n_words': 2410}