<commit_before>TRex Performance Trend
======================
:email: <EMAIL> 
:quotes.++:
:numbered:
:web_server_url: https://trex-tgn.cisco.com/trex
:local_web_server_url: csi-wiki-01:8181/trex
:toclevels: 6
:tabledef-default.subs: normal,callouts

include::trex_ga.asciidoc[]

// PDF version - image width variable
ifdef::backend-docbook[]
:p_width: 450
endif::backend-docbook[]

// HTML version - image width variable
ifdef::backend-xhtml11[]
:p_width: 800
endif::backend-xhtml11[]


= TRex performance trend 

This document tracks the performance of TRex Stateless use-cases of various hardware and NICS.
The intention is to track the TRex software performance and not the NIC driver performance, however from this document you can learn the difference between different driver implementation (cycles/packets)
Performance is measured in a metric called MPPS/Core (Norm), which stands for Mega-Packet per second (Norm). 
 +
How the calculation is made? +
 +
We measure the MPPS of the DUT in its best working point. +
The measurement is then scaled to fit 100% CPU utilization, and finally divided by the number of cores the device has,
to get a normalized MPPS/Core which is denoted as MPPS/Core (Norm) +
 +
How to run these tests? +
[source,python]
----
       trex-core/scripts> ./run_regression --stl -t <name of test> --cfg setups/<name of setup>
----

== Setup: XL710 (4x40GbE)  trex-08

.Setup Details
[options='header',cols="1,5"]
|=================
| Server  | Cisco UCS 240M3 
| CPU:    | 2 sockets x Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 8 cores
| NICs:   | 2 NICS x ( 2x 40GbE ) Intel XL710, Total ports 4
| NUMA    | NIC0-NUMA 0 , NIC1-NUMA1
| Memory  | 2x16GB 
| PCIe    | 2x PCIe16 + 2xPCIe4
| OS:     | Fedora 18 - baremetal
|=================

image:images/trex08_latest_test_runs.png[title="trex08 test runs",align="left",width={p_width}, link="images/trex08_latest_test_runs.png"]
 
[format="csv", options="header",halign='center']
|===
include::build/images/trex08_latest_test_runs_stats.csv[]
|===

=== Trend: Performance Over Time

image:images/trex08_trend_graph.png[title="trex08_trend_graph",align="left",width={p_width}, link="images/trex08_trend_graph.png"]
<commit_msg>added error description to asciidoc<commit_after>TRex Performance Trend
======================
:email: <EMAIL> 
:quotes.++:
:numbered:
:web_server_url: https://trex-tgn.cisco.com/trex
:local_web_server_url: csi-wiki-01:8181/trex
:toclevels: 6
:tabledef-default.subs: normal,callouts

include::trex_ga.asciidoc[]

// PDF version - image width variable
ifdef::backend-docbook[]
:p_width: 450
endif::backend-docbook[]

// HTML version - image width variable
ifdef::backend-xhtml11[]
:p_width: 800
endif::backend-xhtml11[]


= TRex performance trend 

This document tracks the performance of TRex Stateless use-cases of various hardware and NICS.
The intention is to track the TRex software performance and not the NIC driver performance, however from this document you can learn the difference between different driver implementation (cycles/packets)
Performance is measured in a metric called MPPS/Core (Norm), which stands for Mega-Packet per second (Norm). 
 +
How the calculation is made? +
 +
We measure the MPPS of the DUT in its best working point. +
The measurement is then scaled to fit 100% CPU utilization, and finally divided by the number of cores the device has,
to get a normalized MPPS/Core which is denoted as MPPS/Core (Norm) +
 +
How to run these tests? +
[source,python]
----
       trex-core/scripts> ./run_regression --stl -t <name of test> --cfg setups/<name of setup>
----
The Error (%) metric: +
 +
You will see a column titled 'Error (%)' next to other metrics (avg,std etc) +
Error(%) is calculated for each batch of test results +
 +
Taking the Max value (max) and the Min value (min), calculating the difference max-min +
We take the difference and then normalize it to the Average value to get (max-min)/Avg +
Then, we present it as percentage +
 +
Error (%) = ((max-min)/Avg)*100 +
 +

== Setup: XL710 (4x40GbE)  trex-08

.Setup Details
[options='header',cols="1,5"]
|=================
| Server  | Cisco UCS 240M3 
| CPU:    | 2 sockets x Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 8 cores
| NICs:   | 2 NICS x ( 2x 40GbE ) Intel XL710, Total ports 4
| NUMA    | NIC0-NUMA 0 , NIC1-NUMA1
| Memory  | 2x16GB 
| PCIe    | 2x PCIe16 + 2xPCIe4
| OS:     | Fedora 18 - baremetal
|=================

image:images/trex08_latest_test_runs.png[title="trex08 test runs",align="left",width={p_width}, link="images/trex08_latest_test_runs.png"]
 
[format="csv", options="header",halign='center']
|===
include::build/images/trex08_latest_test_runs_stats.csv[]
|===

=== Trend: Performance Over Time

image:images/trex08_trend_graph.png[title="trex08_trend_graph",align="left",width={p_width}, link="images/trex08_trend_graph.png"]

{'fraction_non_alphanumeric': 0.13894042465471038, 'fraction_numerical': 0.034013605442176874, 'mean_word_length': 4.7284533648170015, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1046823', 'n_tokens_mistral': 1764, 'n_tokens_neox': 1601, 'n_words': 512}