<commit_before>.. SPDX-License-Identifier: CC-BY-4.0

Installation and configuration
==============================

Alpine containers have been introduced in Euphrates.
Alpine allows Functest testing in several very light containers and thanks to
the refactoring on dependency management should allow the creation of light and
fully customized docker images.


Functest Dockers for OpenStack deployment
-----------------------------------------
Docker images are available on the dockerhub:

  * opnfv/functest-core
  * opnfv/functest-healthcheck
  * opnfv/functest-smoke
  * opnfv/functest-benchmarking
  * opnfv/functest-features
  * opnfv/functest-components
  * opnfv/functest-vnf


Preparing your environment
^^^^^^^^^^^^^^^^^^^^^^^^^^

cat env::

  EXTERNAL_NETWORK=XXX
  DEPLOY_SCENARIO=XXX  # if not os-nosdn-nofeature-noha scenario
  NAMESERVER=XXX  # if not 8.8.8.8

See section on environment variables for details.

cat env_file::

  export OS_AUTH_URL=XXX
  export OS_USER_DOMAIN_NAME=XXX
  export OS_PROJECT_DOMAIN_NAME=XXX
  export OS_USERNAME=XXX
  export OS_PROJECT_NAME=XXX
  export OS_PASSWORD=<PASSWORD>
  export OS_IDENTITY_API_VERSION=3

See section on OpenStack credentials for details.

Create a directory for the different images (attached as a Docker volume)::

  mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/fraser | bash -s -- images && ls -1 images/*

  images/CentOS-7-aarch64-GenericCloud.qcow2
  images/CentOS-7-aarch64-GenericCloud.qcow2.xz
  images/CentOS-7-x86_64-GenericCloud.qcow2
  images/cirros-0.4.0-x86_64-disk.img
  images/cirros-0.4.0-x86_64-lxc.tar.gz
  images/cloudify-manager-premium-4.0.1.qcow2
  images/shaker-image-arm64.qcow2
  images/shaker-image.qcow
  images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
  images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
  images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
  images/vyos-1.1.7.img

Testing healthcheck suite
^^^^^^^^^^^^^^^^^^^^^^^^^

Run healthcheck suite::

  sudo docker run --env-file env \
      -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
      -v $(pwd)/images:/home/opnfv/functest/images \
      opnfv/functest-healthcheck

Results shall be displayed as follows::

  +----------------------------+------------------+---------------------+------------------+----------------+
  |         TEST CASE          |     PROJECT      |         TIER        |     DURATION     |     RESULT     |
  +----------------------------+------------------+---------------------+------------------+----------------+
  |      connection_check      |     functest     |     healthcheck     |      00:09       |      PASS      |
  |       tenantnetwork1       |     functest     |     healthcheck     |      00:14       |      PASS      |
  |       tenantnetwork2       |     functest     |     healthcheck     |      00:11       |      PASS      |
  |          vmready1          |     functest     |     healthcheck     |      00:19       |      PASS      |
  |          vmready2          |     functest     |     healthcheck     |      00:16       |      PASS      |
  |         singlevm1          |     functest     |     healthcheck     |      00:41       |      PASS      |
  |         singlevm2          |     functest     |     healthcheck     |      00:36       |      PASS      |
  |         vping_ssh          |     functest     |     healthcheck     |      00:46       |      PASS      |
  |       vping_userdata       |     functest     |     healthcheck     |      00:41       |      PASS      |
  |        cinder_test         |     functest     |     healthcheck     |      01:18       |      PASS      |
  |         api_check          |     functest     |     healthcheck     |      10:33       |      PASS      |
  |     snaps_health_check     |     functest     |     healthcheck     |      00:44       |      PASS      |
  |            odl             |     functest     |     healthcheck     |      00:00       |      SKIP      |
  +----------------------------+------------------+---------------------+------------------+----------------+

NOTE: the duration is a reference and it might vary depending on your SUT.

Testing smoke suite
^^^^^^^^^^^^^^^^^^^

Run smoke suite::

  sudo docker run --env-file env \
      -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
      -v $(pwd)/images:/home/opnfv/functest/images \
      opnfv/functest-smoke

Results shall be displayed as follows::

  +------------------------------------+------------------+---------------+------------------+----------------+
  |             TEST CASE              |     PROJECT      |      TIER     |     DURATION     |     RESULT     |
  +------------------------------------+------------------+---------------+------------------+----------------+
  |           tempest_smoke            |     functest     |     smoke     |      04:24       |      PASS      |
  |     neutron-tempest-plugin-api     |     functest     |     smoke     |      10:43       |      PASS      |
  |            rally_sanity            |     functest     |     smoke     |      23:16       |      PASS      |
  |          refstack_defcore          |     functest     |     smoke     |      06:30       |      PASS      |
  |              patrole               |     functest     |     smoke     |      02:08       |      PASS      |
  |            snaps_smoke             |     functest     |     smoke     |      00:00       |      SKIP      |
  |           neutron_trunk            |     functest     |     smoke     |      02:09       |      PASS      |
  |         networking-bgpvpn          |     functest     |     smoke     |      00:00       |      SKIP      |
  |           networking-sfc           |     functest     |     smoke     |      00:00       |      SKIP      |
  |              barbican              |     functest     |     smoke     |      00:00       |      SKIP      |
  +------------------------------------+------------------+---------------+------------------+----------------+

Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.

Testing benchmarking suite
^^^^^^^^^^^^^^^^^^^^^^^^^^

Run benchmarking suite::

  sudo docker run --env-file env \
      -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
      -v $(pwd)/images:/home/opnfv/functest/images \
      opnfv/functest-benchmarking

Results shall be displayed as follows::

  +-------------------+------------------+----------------------+------------------+----------------+
  |     TEST CASE     |     PROJECT      |         TIER         |     DURATION     |     RESULT     |
  +-------------------+------------------+----------------------+------------------+----------------+
  |        vmtp       |     functest     |     benchmarking     |      18:43       |      PASS      |
  |       shaker      |     functest     |     benchmarking     |      29:45       |      PASS      |
  +-------------------+------------------+----------------------+------------------+----------------+

Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.

Testing features suite
^^^^^^^^^^^^^^^^^^^^^^
<commit_msg>Update results in configguide (rally_jobs)<commit_after>.. SPDX-License-Identifier: CC-BY-4.0

Installation and configuration
==============================

Alpine containers have been introduced in Euphrates.
Alpine allows Functest testing in several very light containers and thanks to
the refactoring on dependency management should allow the creation of light and
fully customized docker images.


Functest Dockers for OpenStack deployment
-----------------------------------------
Docker images are available on the dockerhub:

  * opnfv/functest-core
  * opnfv/functest-healthcheck
  * opnfv/functest-smoke
  * opnfv/functest-benchmarking
  * opnfv/functest-features
  * opnfv/functest-components
  * opnfv/functest-vnf


Preparing your environment
^^^^^^^^^^^^^^^^^^^^^^^^^^

cat env::

  EXTERNAL_NETWORK=XXX
  DEPLOY_SCENARIO=XXX  # if not os-nosdn-nofeature-noha scenario
  NAMESERVER=XXX  # if not 8.8.8.8

See section on environment variables for details.

cat env_file::

  export OS_AUTH_URL=XXX
  export OS_USER_DOMAIN_NAME=XXX
  export OS_PROJECT_DOMAIN_NAME=XXX
  export OS_USERNAME=XXX
  export OS_PROJECT_NAME=XXX
  export OS_PASSWORD=<PASSWORD>
  export OS_IDENTITY_API_VERSION=3

See section on OpenStack credentials for details.

Create a directory for the different images (attached as a Docker volume)::

  mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/fraser | bash -s -- images && ls -1 images/*

  images/CentOS-7-aarch64-GenericCloud.qcow2
  images/CentOS-7-aarch64-GenericCloud.qcow2.xz
  images/CentOS-7-x86_64-GenericCloud.qcow2
  images/cirros-0.4.0-x86_64-disk.img
  images/cirros-0.4.0-x86_64-lxc.tar.gz
  images/cloudify-manager-premium-4.0.1.qcow2
  images/shaker-image-arm64.qcow2
  images/shaker-image.qcow
  images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
  images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
  images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
  images/vyos-1.1.7.img

Testing healthcheck suite
^^^^^^^^^^^^^^^^^^^^^^^^^

Run healthcheck suite::

  sudo docker run --env-file env \
      -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
      -v $(pwd)/images:/home/opnfv/functest/images \
      opnfv/functest-healthcheck

Results shall be displayed as follows::

  +----------------------------+------------------+---------------------+------------------+----------------+
  |         TEST CASE          |     PROJECT      |         TIER        |     DURATION     |     RESULT     |
  +----------------------------+------------------+---------------------+------------------+----------------+
  |      connection_check      |     functest     |     healthcheck     |      00:09       |      PASS      |
  |       tenantnetwork1       |     functest     |     healthcheck     |      00:14       |      PASS      |
  |       tenantnetwork2       |     functest     |     healthcheck     |      00:11       |      PASS      |
  |          vmready1          |     functest     |     healthcheck     |      00:19       |      PASS      |
  |          vmready2          |     functest     |     healthcheck     |      00:16       |      PASS      |
  |         singlevm1          |     functest     |     healthcheck     |      00:41       |      PASS      |
  |         singlevm2          |     functest     |     healthcheck     |      00:36       |      PASS      |
  |         vping_ssh          |     functest     |     healthcheck     |      00:46       |      PASS      |
  |       vping_userdata       |     functest     |     healthcheck     |      00:41       |      PASS      |
  |        cinder_test         |     functest     |     healthcheck     |      01:18       |      PASS      |
  |         api_check          |     functest     |     healthcheck     |      10:33       |      PASS      |
  |     snaps_health_check     |     functest     |     healthcheck     |      00:44       |      PASS      |
  |            odl             |     functest     |     healthcheck     |      00:00       |      SKIP      |
  +----------------------------+------------------+---------------------+------------------+----------------+

NOTE: the duration is a reference and it might vary depending on your SUT.

Testing smoke suite
^^^^^^^^^^^^^^^^^^^

Run smoke suite::

  sudo docker run --env-file env \
      -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
      -v $(pwd)/images:/home/opnfv/functest/images \
      opnfv/functest-smoke

Results shall be displayed as follows::

  +------------------------------------+------------------+---------------+------------------+----------------+
  |             TEST CASE              |     PROJECT      |      TIER     |     DURATION     |     RESULT     |
  +------------------------------------+------------------+---------------+------------------+----------------+
  |           tempest_smoke            |     functest     |     smoke     |      06:13       |      PASS      |
  |     neutron-tempest-plugin-api     |     functest     |     smoke     |      09:32       |      PASS      |
  |            rally_sanity            |     functest     |     smoke     |      29:34       |      PASS      |
  |             rally_jobs             |     functest     |     smoke     |      24:02       |      PASS      |
  |          refstack_defcore          |     functest     |     smoke     |      13:07       |      PASS      |
  |              patrole               |     functest     |     smoke     |      05:17       |      PASS      |
  |            snaps_smoke             |     functest     |     smoke     |      90:13       |      PASS      |
  |           neutron_trunk            |     functest     |     smoke     |      00:00       |      SKIP      |
  |         networking-bgpvpn          |     functest     |     smoke     |      00:00       |      SKIP      |
  |           networking-sfc           |     functest     |     smoke     |      00:00       |      SKIP      |
  |              barbican              |     functest     |     smoke     |      05:01       |      PASS      |
  +------------------------------------+------------------+---------------+------------------+----------------+

Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.

Testing benchmarking suite
^^^^^^^^^^^^^^^^^^^^^^^^^^

Run benchmarking suite::

  sudo docker run --env-file env \
      -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
      -v $(pwd)/images:/home/opnfv/functest/images \
      opnfv/functest-benchmarking

Results shall be displayed as follows::

  +-------------------+------------------+----------------------+------------------+----------------+
  |     TEST CASE     |     PROJECT      |         TIER         |     DURATION     |     RESULT     |
  +-------------------+------------------+----------------------+------------------+----------------+
  |        vmtp       |     functest     |     benchmarking     |      18:43       |      PASS      |
  |       shaker      |     functest     |     benchmarking     |      29:45       |      PASS      |
  +-------------------+------------------+----------------------+------------------+----------------+

Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.

Testing features suite
^^^^^^^^^^^^^^^^^^^^^^

{'fraction_non_alphanumeric': 0.22030664395229982, 'fraction_numerical': 0.02385008517887564, 'mean_word_length': 1.0796372396202352, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6531477', 'n_tokens_mistral': 4418, 'n_tokens_neox': 4063, 'n_words': 821}