<commit_before>System(scenario) tests for Sahara project
=========================================

How to run
----------

Scenario framework has default templates for testing Sahara. For
use them, need to specify plugin and version (for transient check and
fake plugin, version is not necessary):

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -p vanilla -v 2.7.1
..

Create the YAML and/or the YAML mako template files for scenario tests
``etc/scenario/sahara-ci/simple-testcase.yaml``.
You can take a look at sample YAML files `How to write scenario files`_.

If you want to run scenario tests for one plugin, you should use the
YAML files with a scenario for the specific plugin:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario etc/scenario/sahara-ci/simple-testcase.yaml
..

or, if the file is a YAML Mako template:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/sahara-ci/vanilla-2.7.1.yaml.mako
..

where templatevars.ini contains the values of the variables referenced
by ``vanilla-2.7.1.yaml.mako``.

For example, you want to run tests for the Vanilla plugin with the Hadoop
version 2.7.1 In this case you should create ``templatevars.ini`` with
the appropriate values (see the section `Variables and sahara-ci templates`_)
and use the following tox env:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/sahara-ci/vanilla-2.7.1.yaml.mako
..

If you want to run scenario tests for a few plugins or their versions, you
should use the several YAML and/or YAML Mako template files:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/sahara-ci/cdh-5.4.0.yaml.mako etc/scenario/sahara-ci/vanilla-2.7.1.yaml.mako ...
..

Here are a few more examples.

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/sahara-ci/credentials.yaml.mako etc/scenario/sahara-ci/vanilla-2.7.1.yaml.mako

..

will run tests for Vanilla plugin with the Hadoop version 2.7.1 and credential
located in ``etc/scenario/sahara-ci/credentials.yaml.mako``, replacing the variables
included into ``vanilla-2.7.1.yaml.mako`` with the values defined into
``templatevars.ini``.
For more information about writing scenario YAML files, see the section
section `How to write scenario files`_.

``tox -e venv -- sahara-scenario etc/scenario/sahara-ci`` will run tests from the test directory.

Also, you can validate your yaml-files using flag ``--validate`` via command:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario --validate -V templatevars.ini etc/scenario/sahara-ci/credantials.yaml.mako etc/scenario/sahara-ci/vanilla-2.7.1.yaml.mako

..

Template variables
------------------
The variables used in the Mako template files are replaced with the values from a
config file, whose name is passed to the test runner through the ``-V`` parameter.

The format of the config file is an INI-style file, as accepted by the Python
ConfigParser module. The key/values must be specified in the DEFAULT section.

Example of template variables file:

.. sourcecode:: ini

    [DEFAULT]
    OS_USERNAME: demo
    OS_TENANT_NAME: demo
    OS_PASSWORD: <PASSWORD>
    ...
    network_type: neutron
    ...

..

Variables and sahara-ci templates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following variables are currently used by sahara-ci templates:<commit_msg>Fix README for sahara-scenario<commit_after>System(scenario) tests for Sahara project
=========================================

How to run
----------

Scenario framework has default templates for testing Sahara. For
use them, need to specify plugin and version (for transient check and
fake plugin, version is not necessary):

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -p vanilla -v 2.7.1
..

Create the YAML and/or the YAML mako template files for scenario tests
``etc/scenario/defaults/simple-testcase.yaml``.
You can take a look at sample YAML files `How to write scenario files`_.

If you want to run scenario tests for one plugin, you should use the
YAML files with a scenario for the specific plugin:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario etc/scenario/defaults/simple-testcase.yaml
..

or, if the file is a YAML Mako template:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/defaults/vanilla-2.7.1.yaml.mako
..

where templatevars.ini contains the values of the variables referenced
by ``vanilla-2.7.1.yaml.mako``.

For example, you want to run tests for the Vanilla plugin with the Hadoop
version 2.7.1 In this case you should create ``templatevars.ini`` with
the appropriate values (see the section `Variables and defaults templates`_)
and use the following tox env:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/defaults/vanilla-2.7.1.yaml.mako
..

If you want to run scenario tests for a few plugins or their versions, you
should use the several YAML and/or YAML Mako template files:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/defaults/cdh-5.4.0.yaml.mako etc/scenario/defaults/vanilla-2.7.1.yaml.mako ...
..

Here are a few more examples.

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario -V templatevars.ini etc/scenario/defaults/credentials.yaml.mako etc/scenario/defaults/vanilla-2.7.1.yaml.mako

..

will run tests for Vanilla plugin with the Hadoop version 2.7.1 and credential
located in ``etc/scenario/defaults/credentials.yaml.mako``, replacing the variables
included into ``vanilla-2.7.1.yaml.mako`` with the values defined into
``templatevars.ini``.
For more information about writing scenario YAML files, see the section
section `How to write scenario files`_.

``tox -e venv -- sahara-scenario etc/scenario/defaults`` will run tests from the test directory.

Also, you can validate your yaml-files using flag ``--validate`` via command:

.. sourcecode:: console

    $ tox -e venv -- sahara-scenario --validate -V templatevars.ini etc/scenario/defaults/credantials.yaml.mako etc/scenario/defaults/vanilla-2.7.1.yaml.mako

..

Template variables
------------------
The variables used in the Mako template files are replaced with the values from a
config file, whose name is passed to the test runner through the ``-V`` parameter.

The format of the config file is an INI-style file, as accepted by the Python
ConfigParser module. The key/values must be specified in the DEFAULT section.

Example of template variables file:

.. sourcecode:: ini

    [DEFAULT]
    OS_USERNAME: demo
    OS_TENANT_NAME: demo
    OS_PASSWORD: <PASSWORD>
    ...
    network_type: neutron
    ...

..

Variables and defaults templates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following variables are currently used by defaults templates:
{'fraction_non_alphanumeric': 0.1193033381712627, 'fraction_numerical': 0.009579100145137881, 'mean_word_length': 3.5216535433070866, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5694791', 'n_tokens_mistral': 2152, 'n_tokens_neox': 2065, 'n_words': 794}