<commit_before># ivy

"ivy" is a Linux-based block storage synthetic workload generator and benchmarking system.

## Features

* **Low CPU overhead** - Written in C++ using Linux native Asynchronous I/O interface to issue multiple I/Os in parallel from one workload thread.
* **Vendor independent design** - LUN attributes decoded by external vendor specific SCSI Inquiry-based LUN discovery tool become selectable in ivy.
  * The Hitachi LUN_discovery toolset is at https://github.com/Hitachi-Data-Systems/LUN_discovery
* **Dynamic feedback control** - PID loop feature enables for example "find IOPS at 1 ms response time".
* **Test in minimum time** - Will run a test as long as necessary to measure to specified plus/minus accuracy.
* **Easy test setup** - Select by vendor LUN attribute such as subsystem logical device, port, parity group, pool ID, etc. instead of by Linux /dev/sdxxx name.
* **Sliced and diced data** - `[CreateRollup] "port";` will print csv files with data rolled up by port.
* **Configuration validation** - `[CreateRollup "port" [quantity] 16 [MaxDroop] "20%";` will invalid results unless 16 ports are reporting and the IOPS for the slowest port is at most 20% below the IOPS for the fastest port.
* **Workflow automation** - `.ivyscript` programming language with "C/C++"-like syntax.

## Vendor proprietary "connectors"

Authorized Hitachi internal users in our labs with the code and license key can use the "command device connector", which retrieves configuration and real-time performance monitor data from Hitachi subsystems.

Other vendors are encouraged to develop their own "connectors" for ivy.

With the Hitachi command device connector, ivy

* Augments the SCSI Inquiry attributes with logical device (LDEV) attributes obtained from the subsystem through the command device.
* Retrieves subsystem performance information synchronized and aligned with ivy host data.
* Filters real-time subsystem data by ivy "rollup instance" enabling fine-grained dynamic feedback control of IOPS to hit a target utilization of a component inside the subsystem.

## Getting started
Look in the doc subfolder and read the "01.Introduction_to_ivy" PowerPoint or .pdf presentation material, or else watch the "Introduction to ivy" video (link below).

## Source code
See src subfolder.  ivy is built using the CodeBlocks IDE.

## Binary Linux executables
The binaries in the bin folder were built on RHEL 6 with g++ 4.9.2 and associated libstdc++.  The binaries are statically linked with libstdc++ and dynamically linked with libc.

I've also built and run ivy on a CentOS 7 platform also with with g++ 4.9.2, with no issues.

## The LUN_discovery tool set is required to run ivy<commit_msg>Update github README.md file with HCPanywhere URL for 3.20.03 binaries<commit_after># ivy

"ivy" is a Linux-based block storage synthetic workload generator and benchmarking system.

## Features

* **Low CPU overhead** - Written in C++ using Linux native Asynchronous I/O interface to issue multiple I/Os in parallel from one workload thread.
* **Vendor independent design** - LUN attributes decoded by external vendor specific SCSI Inquiry-based LUN discovery tool become selectable in ivy.
  * The Hitachi LUN_discovery toolset is at https://github.com/Hitachi-Data-Systems/LUN_discovery
* **Dynamic feedback control** - PID loop feature enables for example "find IOPS at 1 ms response time".
* **Test in minimum time** - Will run a test as long as necessary to measure to specified plus/minus accuracy.
* **Easy test setup** - Select by vendor LUN attribute such as subsystem logical device, port, parity group, pool ID, etc. instead of by Linux /dev/sdxxx name.
* **Sliced and diced data** - `[CreateRollup] "port";` will print csv files with data rolled up by port.
* **Configuration validation** - `[CreateRollup "port" [quantity] 16 [MaxDroop] "20%";` will invalid results unless 16 ports are reporting and the IOPS for the slowest port is at most 20% below the IOPS for the fastest port.
* **Workflow automation** - `.ivyscript` programming language with "C/C++"-like syntax.

## Vendor proprietary "connectors"

Authorized Hitachi internal users in our labs with the code and license key can use the "command device connector", which retrieves configuration and real-time performance monitor data from Hitachi subsystems.

Other vendors are encouraged to develop their own "connectors" for ivy.

With the Hitachi command device connector, ivy

* Augments the SCSI Inquiry attributes with logical device (LDEV) attributes obtained from the subsystem through the command device.
* Retrieves subsystem performance information synchronized and aligned with ivy host data.
* Filters real-time subsystem data by ivy "rollup instance" enabling fine-grained dynamic feedback control of IOPS to hit a target utilization of a component inside the subsystem.

## Getting started
Look in the doc subfolder and read the "01.Introduction_to_ivy" PowerPoint or .pdf presentation material, or else watch the "Introduction to ivy" video (link below).

## Source code
See src subfolder.  ivy is built using the CodeBlocks IDE.

## Binary Linux executables
Download version 3.20.03 from https://hcpanywhere.hitachivantara.com/u/qlXteyNV7QG4Ve16/ivy.3.20.03.bin.zip?l

The binaries for LUN_discovery are included.

These binaries were built on RHEL 7.2.  The binaries are statically linked with libstdc++ and dynamically linked with libc.

## The LUN_discovery tool set is required to run ivy
{'fraction_non_alphanumeric': 0.06768558951965066, 'fraction_numerical': 0.009279475982532752, 'mean_word_length': 4.672858617131063, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2511108', 'n_tokens_mistral': 1522, 'n_tokens_neox': 1409, 'n_words': 758}