<commit_before># DCLedger Load Testing

DCLedger testing is implemented in python3 and based on [Locust](https://locust.io/) framework.

## Requirements

* python >= 3.7
* Install Docker as described at https://docs.docker.com/engine/install/ubuntu/
    - In `Installation methods` section follow `Install using the repository` method
        - Check whether your user of Ubuntu has been added to `docker` group using the following command:

            ```bash
            getent group docker | awk -F: '{print $4}'
            ```

            - If it has not been added, add it using `Manage Docker as a non-root user` section from <https://docs.docker.com/engine/install/linux-postinstall/>
* Install Docker Compose as described at <https://docs.docker.com/compose/install/>

## Installation

Run (consider to use virtual environment):

```bash
pip3 install -r bench/requirements.txt
```

**Optional** If you need to monitor server-side metrics please install [Prometheus](https://prometheus.io/docs/prometheus/latest/getting_started/).

## Preparation

Each write transactions is signed and thus requires:

* an account with write permissions (e.g. Vendor account)
* proper values for txn `sequence` which enforces txns ordering for an account
* Initialize the pool and test accounts (**Warning** applicable to local in-docker pool only for now):

    ```bash
    make localnet_clean

    DCL_OBSERVERS=1 TRUSTEE_ACCOUNT_COUNT=1 make localnet_init

    make localnet_start
    # Note: once started ledger may require some time to complete the initialization.
    ```

* Copy local accounts keys to folder `~/.dcl/keyring-test`:
    ```bash
    cp ./.localnet/node0/keyring-test/* ~/.dcl/keyring-test
    ```
* Enter to `bench` folder
    ```bash
    cd bench
    ```
* Build `docker-compose` file
    ```bash
    docker-compose build
    ```

## Run

### (Optional) Launch Prometheus

```bash
prometheus --config.file=bench/prometheus.yml
```

And open `http://localhost:9090/` to query and monitor the server-side metrics.

## Headless

To run write load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --headless DCLWriteUser --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --headless DCLWriteUser --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.

To run read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --headless DCLReadUser --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --headless DCLReadUser --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.


To run write/read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --headless --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --headless --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.


## Web UI

To run write load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    <br> e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py DCLWriteUser --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py DCLWriteUser --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.

To run read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    <br> e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py DCLReadUser --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py DCLReadUser --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.

To run write/read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    <br> e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.
<commit_msg>Update README.md for running load tests<commit_after># DCLedger Load Testing

DCLedger testing is implemented in python3 and based on [Locust](https://locust.io/) framework.

## Requirements

* python >= 3.7
* Install Docker as described at https://docs.docker.com/engine/install/ubuntu/
    - In `Installation methods` section follow `Install using the repository` method
        - Check whether your user of Ubuntu has been added to `docker` group using the following command:

            ```bash
            getent group docker | awk -F: '{print $4}'
            ```

            - If it has not been added, add it using `Manage Docker as a non-root user` section from <https://docs.docker.com/engine/install/linux-postinstall/>
* Install Docker Compose as described at <https://docs.docker.com/compose/install/>

## Installation

Run (consider to use virtual environment):

```bash
pip3 install -r bench/requirements.txt
```

**Optional** If you need to monitor server-side metrics please install [Prometheus](https://prometheus.io/docs/prometheus/latest/getting_started/).

## Preparation

Each write transactions is signed and thus requires:

* an account with write permissions (e.g. Vendor account)
* proper values for txn `sequence` which enforces txns ordering for an account
* Initialize the pool and test accounts (**Warning** applicable to local in-docker pool only for now):

    ```bash
    make localnet_clean

    make localnet_init

    make localnet_start
    # Note: once started ledger may require some time to complete the initialization.
    ```

* Copy local accounts keys to folder `~/.dcl/keyring-test`:
    ```bash
    cp ./.localnet/node0/keyring-test/* ~/.dcl/keyring-test
    ```
* Enter to `bench` folder
    ```bash
    cd bench
    ```
* Build `docker-compose` file
    ```bash
    docker-compose build
    ```

## Run

### (Optional) Launch Prometheus

```bash
prometheus --config.file=bench/prometheus.yml
```

And open `http://localhost:9090/` to query and monitor the server-side metrics.

## Headless

To run write load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --headless WriteModelLoadTest --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --headless WriteModelLoadTest --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.

To run read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --headless ReadModelLoadTest --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --headless ReadModelLoadTest --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.


To run write/read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --headless --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --headless --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.


## Web UI

To run write load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    <br> e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py WriteModelLoadTest --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py WriteModelLoadTest --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.

To run read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    <br> e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py ReadModelLoadTest --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py ReadModelLoadTest --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.

To run write/read load tests

* Open the docker-compose.yml and update field `command` for `master` and `worker`.  
    <br> e.g.: for `master`
    ```yml
    command: -f /dcl/bench/locustfile.py --master -H http://master:8089 
    ```
    e.g.: for `worker`:
    ```yml
    command: -f /dcl/bench/locustfile.py --worker --master-host master
    ```

* Run docker-compose.yml
    ```bash
    docker-compose up --scale worker=<workers-count>
    # The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.
    ```
    `<workers-count>`- number of machine.

{'fraction_non_alphanumeric': 0.11559675195342424, 'fraction_numerical': 0.005362341044890455, 'mean_word_length': 3.0231124807395995, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 39, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1148714', 'n_tokens_mistral': 4083, 'n_tokens_neox': 3805, 'n_words': 1475}