<commit_before># Aqua-lang

Aqua-lang data processing language inspired by [Kusto(Azure Log Analytics)](https://docs.loganalytics.io).

## Supported datatypes

- int32
- int64
- float64
- char
- string

## EBNF

```ebnf
statement = relation , { "|" , operator } ;
relation = "test" ;
<Statement> ::= <Relation> {"|" <Operator>}
<Operator>  ::= <OrderByOperator>
              | <WhereOperator>
              | <ProjectOperator>
              | <DistinctOperator>
              | <TakeOperator>
              | <JoinOperator>
              | <UnionOperator>
```

## Order by

```ebnf
<OrderByOperator>     ::= "order by" <AttributeOrderPair> {"," <AttributeOrderPair>}
<AttributeOrderPair>  ::= <Attribute> ["asc" | "desc"]
```

Example<commit_msg>Convert some specification to EBNF<commit_after># Aqua-lang

Aqua-lang data processing language inspired by [Kusto(Azure Log Analytics)](https://docs.loganalytics.io).

## Supported datatypes

- int32
- int64
- float64
- char
- string

## EBNF

```ebnf
statement = relation , { "|" , operator }
operator  = order by operator
          | where operator
          | project operator
          | distinct operator
          | take operator
          | join operator
          | union operator
<Statement> ::= <Relation> {"|" <Operator>}
<Operator>  ::= <OrderByOperator>
              | <WhereOperator>
              | <ProjectOperator>
              | <DistinctOperator>
              | <TakeOperator>
              | <JoinOperator>
              | <UnionOperator>
```

## Order by

```ebnf
<OrderByOperator>     ::= "order by" <AttributeOrderPair> {"," <AttributeOrderPair>}
<AttributeOrderPair>  ::= <Attribute> ["asc" | "desc"]
```

Example
{'fraction_non_alphanumeric': 0.14691943127962084, 'fraction_numerical': 0.0071090047393364926, 'mean_word_length': 2.6400862068965516, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5719325', 'n_tokens_mistral': 527, 'n_tokens_neox': 504, 'n_words': 115}