<commit_before># Clack - Lisp (Clojurescript) on the command line.

Clack is a utility for destructuring and transforming data structures on the command line. It runs as a clojurescript application on the Node.js VM, and allows for the application of clojurescript functions to data within shell (eg. bash) pipelines.

Clack can be thought of as a replacement to sed, awk, grep etc. Where these tools have thrived in processing tabular, line based data over the decaded, clack targets structured data such as JSON, YAML, TransitJs and EDN - although it can be useed as a shell line processor as well.

Clack addresses the same problem spave as the excellent [jq](https://stedolan.github.io/jq/), however where that project focuses exclusively on JSON data, clack targets many data formats.

## Installation

```
npm install -g clack-cljs
```

## First look

Clack transforms input data from STDIN into Clojure (EDN) data structures, applies clojure functions to this data, and prints it to STDOUT. The true power of this process is in the Clojure functions themselves, so the interface to the clack binary is very simple.

Let's look at a trivial example. We have a JSON data structure in a file:

```
{
  "foo": "bar", 
  "baz": {
    "its": [
      "very", 
      "deep"
    ]
  }
}
```

In most line processors it would be hard to extract data nested inside this map. In clack, not so. We can do:

```
$ cat file.json | clack -i js :baz :its -e second
"deep"
```

To break this down:

`-i js` - treat the input as JSON. This input is converted into in memory clojurescript data structures on which our transformations will be applied.

`:baz :its -e second` - apply these tramsformations to the data in left to right order.

So `:baz` and `:its` are clojurescript keywords, that will descend down through the keys of the nested maps.

And `-e second` will evaluate the clojurescript `eval` function, the second element of the array.

## Usage

The top-level switches for clack are:

`clack [-i input-format] [-o output-format] [-w] [-u] [transformation pipeline ...]`

Input/ouput formats can be any one of edn (default), js (json), yaml, tjs (transit) or s (string).

### The trasformation pipeline

The pipeline will treat tokens as follows:

`:foo` - any token prefixed with a colon is interpreted as a clojurescript keyword.

```
$ echo {:foo :x} | clack :foo
:x
```

`1` - any number will be interpreted as a number

```<commit_msg>Typo: transformation<commit_after># Clack - Lisp (Clojurescript) on the command line.

Clack is a utility for destructuring and transforming data structures on the command line. It runs as a clojurescript application on the Node.js VM, and allows for the application of clojurescript functions to data within shell (eg. bash) pipelines.

Clack can be thought of as a replacement to sed, awk, grep etc. Where these tools have thrived in processing tabular, line based data over the decaded, clack targets structured data such as JSON, YAML, TransitJs and EDN - although it can be useed as a shell line processor as well.

Clack addresses the same problem spave as the excellent [jq](https://stedolan.github.io/jq/), however where that project focuses exclusively on JSON data, clack targets many data formats.

## Installation

```
npm install -g clack-cljs
```

## First look

Clack transforms input data from STDIN into Clojure (EDN) data structures, applies clojure functions to this data, and prints it to STDOUT. The true power of this process is in the Clojure functions themselves, so the interface to the clack binary is very simple.

Let's look at a trivial example. We have a JSON data structure in a file:

```
{
  "foo": "bar", 
  "baz": {
    "its": [
      "very", 
      "deep"
    ]
  }
}
```

In most line processors it would be hard to extract data nested inside this map. In clack, not so. We can do:

```
$ cat file.json | clack -i js :baz :its -e second
"deep"
```

To break this down:

`-i js` - treat the input as JSON. This input is converted into in memory clojurescript data structures on which our transformations will be applied.

`:baz :its -e second` - apply these tramsformations to the data in left to right order.

So `:baz` and `:its` are clojurescript keywords, that will descend down through the keys of the nested maps.

And `-e second` will evaluate the clojurescript `eval` function, the second element of the array.

## Usage

The top-level switches for clack are:

`clack [-i input-format] [-o output-format] [-w] [-u] [transformation pipeline ...]`

Input/ouput formats can be any one of edn (default), js (json), yaml, tjs (transit) or s (string).

### The transformation pipeline

The pipeline will treat tokens as follows:

`:foo` - any token prefixed with a colon is interpreted as a clojurescript keyword.

```
$ echo {:foo :x} | clack :foo
:x
```

`1` - any number will be interpreted as a number

```
{'fraction_non_alphanumeric': 0.08367430441898527, 'fraction_numerical': 0.0004091653027823241, 'mean_word_length': 3.821499013806706, 'pattern_counts': {'":': 6, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7332055', 'n_tokens_mistral': 1465, 'n_tokens_neox': 1388, 'n_words': 745}