<commit_before>// Module included in the following assemblies:
// Epic CNF-290 (4.5)
// scalability_and_performance/cnf-performance-addon-operator-for-low-latency-nodes.adoc

[id="cnf-performing-end-to-end-tests-for-platform-verification_{context}"]
= Performing end-to-end tests for platform verification

The Cloud-native Network Functions link:https://quay.io/repository/openshift-kni/cnf-tests?tag=latest&tab=tags[(CNF)] tests image is a containerized test suite that validates features required to run CNF payloads. You can use this image to validate a CNF-enabled OpenShift cluster where all the components required for running CNF workloads are installed.

The tests run by the image are split into three different phases:

* Simple cluster validation
* Setup
* End to end tests

The validation phase checks that all the features required to be tested are deployed correctly on the cluster.

Validations include:

* Targeting a machine config pool that belong to the machines to be tested
* Enabling SCTP on the nodes
* Having the Performance Addon Operator installed
* Having the SR-IOV Operator installed
* Having the PTP Operator installed
* Using OVN kubernetes as the SDN

Latency tests, a part of the CNF-test container, also require the same validations. For more information about running a latency test, see the Running the latency tests section.

The tests need to perform an environment configuration every time they are executed. This involves items such as creating SRI-OV node policies, performance profiles, or PTP profiles. Allowing the tests to configure an already configured cluster might affect the functionality of the cluster. Also, changes to configuration items such as SR-IOV node policy might result in the environment being temporarily unavailable until the configuration change is processed.

[id="cnf-performing-end-to-end-tests-prerequisites_{context}"]
== Prerequisites

* The test entrypoint is `/usr/bin/test-run.sh`. It runs both a setup test set and the real conformance test suite. The minimum requirement is to provide it with a kubeconfig file and its related `$KUBECONFIG` environment variable, mounted through a volume.

* The tests assumes that a given feature is already available on the cluster in the form of an Operator, flags enabled on the cluster, or machine configs.

* Some tests require a pre-existing machine config pool to append their changes to. This must be created on the cluster before running the tests.
+
The default worker pool is `worker-cnf` and can be created with the following manifest:
+
[source,yaml]
----
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
  name: worker-cnf
  labels:
    machineconfiguration.openshift.io/role: worker-cnf
spec:
  machineConfigSelector:
    matchExpressions:
      - {
          key: machineconfiguration.openshift.io/role,
          operator: In,
          values: [worker-cnf, worker],
        }
  paused: false
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/worker-cnf: ""
----
+
You can use the `ROLE_WORKER_CNF` variable to override the worker pool name:
+
[source,terminal]
----
$ docker run -v $(pwd)/:/kubeconfig -e KUBECONFIG=/kubeconfig/kubeconfig -e
ROLE_WORKER_CNF=custom-worker-pool registry.redhat.io/openshift4/cnf-tests-rhel8:v4.6 /usr/bin/test-run.sh
----
+
[NOTE]
====
Currently, not all tests run selectively on the nodes belonging to the pool.
====

[id="cnf-performing-end-to-end-tests-running-the-tests_{context}"]
== Running the tests
Assuming the `kubeconfig` file is in the current folder, the command for running the test suite is:

----
$ docker run -v $(pwd)/:/kubeconfig -e KUBECONFIG=/kubeconfig/kubeconfig registry.redhat.io/openshift4/cnf-tests-rhel8:v4.6 /usr/bin/test-run.sh
----

This allows your `kubeconfig` file to be consumed from inside the running container.

[id="cnf-performing-end-to-end-tests-running-the-latency_tests_{context}"]
=== Running the latency tests
In {product-title} {product-version}, you can also run latency tests from the CNF-test container. The latency test allows you to set a latency limit so that you can determine performance, throughput, and latency.

The latency test runs the `oslat` tool, which is an open source program to detect OS level latency. For more information, see the Red Hat Knowledgebase solution link:https://access.redhat.com/solutions/5315541[How to measure OS and hardware latency on isolated CPUs?].

By default, the latency tests are disabled. To enable the latency test, you must add the `LATENCY_TEST_RUN` variable and set its value to `true`. For example, `LATENCY_TEST_RUN=true`.

Additionally, you can set the following environment variables for latency tests:

* `LATENCY_TEST_RUNTIME` - Specifies the amount of time (in seconds) that the latency test must run.
* `OSLAT_MAXIMUM_LATENCY` - Specifies the maximum latency (in microseconds) that is expected from all buckets during the `oslat` test run.

To perform the latency tests, run the following command:

----
$ docker run -v $(pwd)/:/kubeconfig -e KUBECONFIG=/kubeconfig/kubeconfig -e LATENCY_TEST_RUN=true -e LATENCY_TEST_RUNTIME=600 -e OSLAT_MAXIMUM_LATENCY=20 registry.redhat.io/openshift4/cnf-tests-rhel8:v4.7 /usr/bin/test-run.sh
----
[NOTE]
====
You must run the latency test in discovery mode. For more information, see the Discovery mode section.
====

.Excerpt of a sample result of a 10-second latency test using the following command:
[source,terminal]
----
[root@cnf12-installer ~]# podman run --rm -v $KUBECONFIG:/kubeconfig:Z -e PERF_TEST_PROFILE=worker-cnf-2 -e KUBECONFIG=/kubeconfig -e LATENCY_TEST_RUN=true -e LATENCY_TEST_RUNTIME=10 -e OSLAT_MAXIMUM_LATENCY=20 -e DISCOVERY_MODE=true registry.redhat.io/openshift4/cnf-tests-rhel8:v4.7 /usr/bin/test-run.sh<commit_msg>Updated the link ID to resolve the RN linking issue<commit_after>// Module included in the following assemblies:
// Epic CNF-290 (4.5)
// scalability_and_performance/cnf-performance-addon-operator-for-low-latency-nodes.adoc

[id="cnf-performing-end-to-end-tests-for-platform-verification_{context}"]
= Performing end-to-end tests for platform verification

The Cloud-native Network Functions link:https://quay.io/repository/openshift-kni/cnf-tests?tag=latest&tab=tags[(CNF)] tests image is a containerized test suite that validates features required to run CNF payloads. You can use this image to validate a CNF-enabled OpenShift cluster where all the components required for running CNF workloads are installed.

The tests run by the image are split into three different phases:

* Simple cluster validation
* Setup
* End to end tests

The validation phase checks that all the features required to be tested are deployed correctly on the cluster.

Validations include:

* Targeting a machine config pool that belong to the machines to be tested
* Enabling SCTP on the nodes
* Having the Performance Addon Operator installed
* Having the SR-IOV Operator installed
* Having the PTP Operator installed
* Using OVN kubernetes as the SDN

Latency tests, a part of the CNF-test container, also require the same validations. For more information about running a latency test, see the Running the latency tests section.

The tests need to perform an environment configuration every time they are executed. This involves items such as creating SRI-OV node policies, performance profiles, or PTP profiles. Allowing the tests to configure an already configured cluster might affect the functionality of the cluster. Also, changes to configuration items such as SR-IOV node policy might result in the environment being temporarily unavailable until the configuration change is processed.

[id="cnf-performing-end-to-end-tests-prerequisites_{context}"]
== Prerequisites

* The test entrypoint is `/usr/bin/test-run.sh`. It runs both a setup test set and the real conformance test suite. The minimum requirement is to provide it with a kubeconfig file and its related `$KUBECONFIG` environment variable, mounted through a volume.

* The tests assumes that a given feature is already available on the cluster in the form of an Operator, flags enabled on the cluster, or machine configs.

* Some tests require a pre-existing machine config pool to append their changes to. This must be created on the cluster before running the tests.
+
The default worker pool is `worker-cnf` and can be created with the following manifest:
+
[source,yaml]
----
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
  name: worker-cnf
  labels:
    machineconfiguration.openshift.io/role: worker-cnf
spec:
  machineConfigSelector:
    matchExpressions:
      - {
          key: machineconfiguration.openshift.io/role,
          operator: In,
          values: [worker-cnf, worker],
        }
  paused: false
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/worker-cnf: ""
----
+
You can use the `ROLE_WORKER_CNF` variable to override the worker pool name:
+
[source,terminal]
----
$ docker run -v $(pwd)/:/kubeconfig -e KUBECONFIG=/kubeconfig/kubeconfig -e
ROLE_WORKER_CNF=custom-worker-pool registry.redhat.io/openshift4/cnf-tests-rhel8:v4.6 /usr/bin/test-run.sh
----
+
[NOTE]
====
Currently, not all tests run selectively on the nodes belonging to the pool.
====

[id="cnf-performing-end-to-end-tests-running-the-tests_{context}"]
== Running the tests
Assuming the `kubeconfig` file is in the current folder, the command for running the test suite is:

----
$ docker run -v $(pwd)/:/kubeconfig -e KUBECONFIG=/kubeconfig/kubeconfig registry.redhat.io/openshift4/cnf-tests-rhel8:v4.6 /usr/bin/test-run.sh
----

This allows your `kubeconfig` file to be consumed from inside the running container.

[id="cnf-performing-end-to-end-tests-running-the-latency_tests"]
=== Running the latency tests
In {product-title} {product-version}, you can also run latency tests from the CNF-test container. The latency test allows you to set a latency limit so that you can determine performance, throughput, and latency.

The latency test runs the `oslat` tool, which is an open source program to detect OS level latency. For more information, see the Red Hat Knowledgebase solution link:https://access.redhat.com/solutions/5315541[How to measure OS and hardware latency on isolated CPUs?].

By default, the latency tests are disabled. To enable the latency test, you must add the `LATENCY_TEST_RUN` variable and set its value to `true`. For example, `LATENCY_TEST_RUN=true`.

Additionally, you can set the following environment variables for latency tests:

* `LATENCY_TEST_RUNTIME` - Specifies the amount of time (in seconds) that the latency test must run.
* `OSLAT_MAXIMUM_LATENCY` - Specifies the maximum latency (in microseconds) that is expected from all buckets during the `oslat` test run.

To perform the latency tests, run the following command:

----
$ docker run -v $(pwd)/:/kubeconfig -e KUBECONFIG=/kubeconfig/kubeconfig -e LATENCY_TEST_RUN=true -e LATENCY_TEST_RUNTIME=600 -e OSLAT_MAXIMUM_LATENCY=20 registry.redhat.io/openshift4/cnf-tests-rhel8:v4.7 /usr/bin/test-run.sh
----
[NOTE]
====
You must run the latency test in discovery mode. For more information, see the Discovery mode section.
====

.Excerpt of a sample result of a 10-second latency test using the following command:
[source,terminal]
----
[root@cnf12-installer ~]# podman run --rm -v $KUBECONFIG:/kubeconfig:Z -e PERF_TEST_PROFILE=worker-cnf-2 -e KUBECONFIG=/kubeconfig -e LATENCY_TEST_RUN=true -e LATENCY_TEST_RUNTIME=10 -e OSLAT_MAXIMUM_LATENCY=20 -e DISCOVERY_MODE=true registry.redhat.io/openshift4/cnf-tests-rhel8:v4.7 /usr/bin/test-run.sh
{'fraction_non_alphanumeric': 0.0838209982788296, 'fraction_numerical': 0.007401032702237522, 'mean_word_length': 4.455868544600939, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6615891', 'n_tokens_mistral': 3511, 'n_tokens_neox': 3354, 'n_words': 1340}