<commit_before># influent-benchmark

Build servers to test Influent and Fluentd.

## Setup

### Receiver node

```
sudo yum -y install git
git clone https://github.com/okumin/influent-benchmark.git ~/influent-benchmark
~/influent-benchmark/setup_receiver.sh
source ~/.bash_profile
```

### Sender node

```
RECEIVER_HOST={host name or ip address of target host}
sudo yum -y install git
git clone https://github.com/okumin/influent-benchmark.git ~/influent-benchmark
~/influent-benchmark/setup_sender.sh ${RECEIVER_HOST}
source ~/.bash_profile
```

## Benchmark

### 1. Start the receiver

When testing Fluentd 0.12,

```
cd ~/fluentd-benchmark-0.12/in_forward
bundle exec fluentd -c receiver.conf
```

When testing Fluentd 0.14,

```
cd ~/fluentd-benchmark-0.14/in_forward
bundle exec fluentd -c receiver.conf
```

When testing Influent,

```
~/influent/start_influent.sh {test branch}
```

### 2. Start benchmark

* protocol: 0.12 or 0.14
* mode: at-most-once or at-least-once
* message/sec: message per sec

```
~/influent-benchmark/run_benchmark.sh {protocol} {mode} {message/sec}
```
<commit_msg>Fix README<commit_after># influent-benchmark

Build servers to test Influent and Fluentd.

## Setup

### Receiver node

```
sudo yum -y install git
git clone https://github.com/okumin/influent-benchmark.git ~/influent-benchmark
~/influent-benchmark/setup_receiver.sh
source ~/.bash_profile
```

### Sender node

```
RECEIVER_HOST={host name or ip address of target host}
sudo yum -y install git
git clone https://github.com/okumin/influent-benchmark.git ~/influent-benchmark
~/influent-benchmark/setup_sender.sh ${RECEIVER_HOST}
source ~/.bash_profile
```

## Benchmark

### 1. Start the receiver

When testing Fluentd 0.12,

```
cd ~/fluentd-benchmark-0.12/in_forward
bundle exec fluentd -c receiver.conf
```

When testing Fluentd 0.14,

```
cd ~/fluentd-benchmark-0.14/in_forward
bundle exec fluentd -c receiver.conf
```

When testing Influent,

```
~/influent/start_influent.sh {test branch}
```

### 2. Start benchmark

* version: 0.12 or 0.14
* mode: at-most-once or at-least-once
* message/sec: message per sec

```
~/influent-benchmark/run_benchmark.sh {version} {mode} {message/sec}
```

{'fraction_non_alphanumeric': 0.14396355353075171, 'fraction_numerical': 0.018223234624145785, 'mean_word_length': 4.35609756097561, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '693179', 'n_tokens_mistral': 855, 'n_tokens_neox': 798, 'n_words': 199}