<commit_before>In MZBench, scenarios are .bdl files written in a special DSL (domain specific language). BDL stands for Benchmark Definition Language. Think of it as a simple ident-based (like python) language with a small set of instructions and measurement units.

MZBench test scenarios consist of function calls and multi-line statements. Function name is *identifier*. Indetifier is lower-case letter sequence with numbers and underscore which starts from letter. Function could accept positional arguments or key arguments. Position arguments are values, key arguments are keys with values, for example:

```python
multiline(param1 = 10, param2 = 20):
    function1(1, 2)
    function2(param1 = function3(1), param2 = 2)
    ...
```

Function value could be used in some cases, in the example above `function3` value is used to pass to function2.

Some statements only appear at the top level of a scenario. They're called *top-level statements*. There're two kinds of top-level statements: [directives](#directives) and [pools](#pools).

[See live examples of MZBench scenarios on GitHub →](https://github.com/machinezone/mzbench/tree/master/examples.bdl)


# Directives

**Directives** prepare the system for the benchmark and clean up after. It includes installing an external [worker](../workers.md) on test nodes, registering resource files, checking conditions, and executing shell commands before and after the test.

## Top-Level Directives

All top-level directives are optional.

### make_install git

```python
make_install(git = "<URL>", branch = "<Branch>", dir = "<Dir>")
```

Install an external worker from a remote git repository on the test nodes before running the benchmark.

MZBench downloads the worker and builds a .tgz archive, which is then distributed among the nodes and used in future provisions.

The following actions are executed during `make_install`:

```bash
$ git clone <URL> temp_dir
$ cd temp_dir
$ git checkout <Branch>
$ cd <Dir>
$ make generate_tgz
```

If `branch` is not specified, the default git branch is used.

If `dir` is not specified, `.` is used.

### make_install rsync

```python
make_install(rsync = "<location>", exclude = "<subdir>")
```

Install an external worker with rsync on the test nodes before running the benchmark. Unlike git `make_install`, rsync does not cache worker code.

### defaults

```python
defaults("<VarName1>" = <Value1>, "<VarName2>" = <Value2>, ...)
```

Allows to define the default values for environment variables, i.e. the values used if no value was provided for this variable on the command line.

See [Environment Variables](#environment-variables) for additional information.

### include_resource

```python
include_resource(<ResourceName>, "<FileName>", <Type>)
include_resource(<ResourceName>, "<FileURL>", <Type>)
```

Register a [resource file](#resource-files) as `<ResourceName>`.

If the file is on your local machine, put it in the same directory where you invoke `mzbench run`.

**`<Type>`** is one of the following atoms:

`text`
:   Plain text file, interpreted as a single string.

`json`
:   JSON file. Lists are interpreted as [Erlang lists](http://www.erlang.org/doc/man/lists.html), objects are interpreted as [Erlang maps](http://www.erlang.org/doc/man/maps.html).

`tsv`
:   File with tabulation separated values, interpreted as a list of lists.

`erlang`
:    Erlang source file, interpreted directly as an [Erlang term](http://erlang.org/doc/reference_manual/expressions.html#id77790).

`binary`
:   Custom binary (image, executable, archive, etc.), not interpreted.<commit_msg>include_resource doc correction<commit_after>In MZBench, scenarios are .bdl files written in a special DSL (domain specific language). BDL stands for Benchmark Definition Language. Think of it as a simple ident-based (like python) language with a small set of instructions and measurement units.

MZBench test scenarios consist of function calls and multi-line statements. Function name is *identifier*. Indetifier is lower-case letter sequence with numbers and underscore which starts from letter. Function could accept positional arguments or key arguments. Position arguments are values, key arguments are keys with values, for example:

```python
multiline(param1 = 10, param2 = 20):
    function1(1, 2)
    function2(param1 = function3(1), param2 = 2)
    ...
```

Function value could be used in some cases, in the example above `function3` value is used to pass to function2.

Some statements only appear at the top level of a scenario. They're called *top-level statements*. There're two kinds of top-level statements: [directives](#directives) and [pools](#pools).

[See live examples of MZBench scenarios on GitHub →](https://github.com/machinezone/mzbench/tree/master/examples.bdl)


# Directives

**Directives** prepare the system for the benchmark and clean up after. It includes installing an external [worker](../workers.md) on test nodes, registering resource files, checking conditions, and executing shell commands before and after the test.

## Top-Level Directives

All top-level directives are optional.

### make_install git

```python
make_install(git = "<URL>", branch = "<Branch>", dir = "<Dir>")
```

Install an external worker from a remote git repository on the test nodes before running the benchmark.

MZBench downloads the worker and builds a .tgz archive, which is then distributed among the nodes and used in future provisions.

The following actions are executed during `make_install`:

```bash
$ git clone <URL> temp_dir
$ cd temp_dir
$ git checkout <Branch>
$ cd <Dir>
$ make generate_tgz
```

If `branch` is not specified, the default git branch is used.

If `dir` is not specified, `.` is used.

### make_install rsync

```python
make_install(rsync = "<location>", exclude = "<subdir>")
```

Install an external worker with rsync on the test nodes before running the benchmark. Unlike git `make_install`, rsync does not cache worker code.

### defaults

```python
defaults("<VarName1>" = <Value1>, "<VarName2>" = <Value2>, ...)
```

Allows to define the default values for environment variables, i.e. the values used if no value was provided for this variable on the command line.

See [Environment Variables](#environment-variables) for additional information.

### include_resource

```python
include_resource(<ResourceName>, "<FileName>", <Type>)
include_resource(<ResourceName>, "<FileURL>", <Type>)
```

Register a [resource file](#resource-files) as `<ResourceName>`.

If the file is on your local machine, put it in the same directory where your scenario is.

**`<Type>`** is one of the following atoms:

`text`
:   Plain text file, interpreted as a single string.

`json`
:   JSON file. Lists are interpreted as [Erlang lists](http://www.erlang.org/doc/man/lists.html), objects are interpreted as [Erlang maps](http://www.erlang.org/doc/man/maps.html).

`tsv`
:   File with tabulation separated values, interpreted as a list of lists.

`erlang`
:    Erlang source file, interpreted directly as an [Erlang term](http://erlang.org/doc/reference_manual/expressions.html#id77790).

`binary`
:   Custom binary (image, executable, archive, etc.), not interpreted.
{'fraction_non_alphanumeric': 0.10255697609783213, 'fraction_numerical': 0.007226236798221234, 'mean_word_length': 4.794685990338165, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 2, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7110485', 'n_tokens_mistral': 2026, 'n_tokens_neox': 1927, 'n_words': 880}