<commit_before># SimpleMapReduce

Simple MapReduce implementation, written in JavaScript.

## Installation

Via npm on Node:

```
npm install simplemapreduce
```


## Usage

Reference in your program:

```js
var simplemapreduce = require('simplemapreduce');
```

### Synchronous Map Reduce

Synchronous run
```js
simplemapreduce.mapReduceSync(items, mapfn, reducefn);
```
where

- `items`: to be processed. In the current version, it's an object with `forEach` function defined.
- `mapfn(key, value, ctx)`: given a key/value pair, it emits zero, one or more key/value pairs using `ctx`.
See example below.
- `reducefn(key, values, ctx)`: given a key and its associated values, emits zero, one or more key/value pairs using `ctx`.

Word count example
```js
var result = simplemapreduce.mapReduceSync(
    ["A", "word", "is", "a", "word"], // items to process
    function (key, value, ctx) { ctx.emit(value.toLowerCase(), 1); }, // map
    function (key, values, ctx) { // reduce
        var total = 0;
        values.forEach(function (value) {
            total += value;
        });
        ctx.emit(key, total);
    }
);

// result.a === 2
// result.word === 2
// result.is === 1
```

### Run

Synchronous run
```js
simplemapreduce.runSync(items, mapfn, newfn, processfn);
```
where

- `items`: to be processed. In the current version, it's an object with `forEach` function defined.
- `mapfn(item)`: given an item to be processed, it returns it's associated key..
- `newfn(item, key)`: given a new key, it returns the new object to be associated with that key.
- `processfn(item, result, [key, map])`: process an item, usually modifying its associated result object.
In addition, it could receive and use the associated key and the `map`, the dictionary that is being build by the
process.

Example<commit_msg>async mapReduce in README.md<commit_after># SimpleMapReduce

Simple MapReduce implementation, written in JavaScript.

## Installation

Via npm on Node:

```
npm install simplemapreduce
```


## Usage

Reference in your program:

```js
var simplemapreduce = require('simplemapreduce');
```

### Synchronous Map Reduce

Synchronous run
```js
simplemapreduce.mapReduceSync(items, mapfn, reducefn);
```
where

- `items`: to be processed. In the current version, it's an object with `forEach` function defined.
- `mapfn(key, value, ctx)`: given a key/value pair, it emits zero, one or more key/value pairs using `ctx`.
See example below.
- `reducefn(key, values, ctx)`: given a key and its associated values, emits zero, one or more key/value pairs using `ctx`.

Word count example
```js
var result = simplemapreduce.mapReduceSync(
    ["A", "word", "is", "a", "word"], // items to process
    function (key, value, ctx) { ctx.emit(value.toLowerCase(), 1); }, // map
    function (key, values, ctx) { // reduce
        var total = 0;
        values.forEach(function (value) {
            total += value;
        });
        ctx.emit(key, total);
    }
);

```

### Map Reduce
```js
simplemapreduce.mapReduceSync(items, mapfn, reducefn, callbackfn);
```
where

- `items`: to be processed. In the current version, it's an object with keys that can be obtained using `Object.keys(items)`.
- `mapfn(key, value, ctx, next)`: given a key/value pair, it emits zero, one or more key/value pairs using `ctx`.
See example below.
- `reducefn(key, values, ctx, next)`: given a key and its associated values, emits zero, one or more key/value pairs using `ctx`.
- `callbackfn(err, results)`: `results` results is a dictionary with the key/value reduce outcome.

```js
simplemapreduce.mapReduce(
    ["A", "word", "is", "a", "word"], // items to process
    function (key, value, ctx, next) { ctx.emit(value.toLowerCase(), 1); next(); }, // map
    function (key, values, ctx, next) { // reduce
        var total = 0;
        values.forEach(function (value) {
            total += value;
        });
        ctx.emit(key, total);
        next();
    },
    function (err, result) {
		// result.a === 2
		// result.word === 2
		// result.is === 1
    }
);
```
The above example is simple: it is executed without doing async calls in the map/reduce functions. But you
can call next callback at any time.

### Run

Synchronous run
```js
simplemapreduce.runSync(items, mapfn, newfn, processfn);
```
where

- `items`: to be processed. In the current version, it's an object with `forEach` function defined.
- `mapfn(item)`: given an item to be processed, it returns it's associated key..
- `newfn(item, key)`: given a new key, it returns the new object to be associated with that key.
- `processfn(item, result, [key, map])`: process an item, usually modifying its associated result object.
In addition, it could receive and use the associated key and the `map`, the dictionary that is being build by the
process.

Example
{'fraction_non_alphanumeric': 0.133236333402619, 'fraction_numerical': 0.002494283932654334, 'mean_word_length': 3.626923076923077, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '695656', 'n_tokens_mistral': 1504, 'n_tokens_neox': 1435, 'n_words': 585}