<commit_before># Underscore.aggregate

The aggregation pipeline is a framework for data aggregation that make your code easier to read and understand. 

Collection's items (objects) enter a multi-stage pipeline that transforms them and the collection itself into an aggregated results.

__Basic example:__
```js
_([
    { level: 'warn',  date: '1999-12-31 23:59', content: 'dates may be a problem...' },
    { level: 'debug', date: '2000-01-01 00:00', content: 'everything seems to be fine...' },
    { level: 'info',  date: '2000-01-01 01:00', content: 'current date is...' },
    { level: 'warn',  date: '2000-01-01 02:00', content: 'possible memory leak...' },
    { level: 'error', date: '2000-01-01 03:00', content: 'throw an error during date conversion...' },
    { level: 'error', date: '2000-01-01 03:30', content: 'throw a new error during date conversion...' },
    { level: 'info',  date: '2000-01-01 04:00', content: 'everything seems ok now...' }
]).aggregate([
    {  
        $project: { // transform items for the next stage
            level: 1, // <=> level : '$level'
            date: { $parse: '$date' }, // convert string to date
            content: 1
        }
    },
    {
        $match: { // filter items that match following conditions
            level: { $in: ['warn', 'error'] },
            date: {
                $gte: moment('2000-01-01 00:00'),
                $lt: moment('2000-01-02 00:00')
            },
            content: { $regex: /.*date.*/} // filter items whose content contains the word "date"
        }
    },
    {
        $group: { // group remaining items by level and compute aggregated infos
            _id: '$level',
            count: { $sum: 1 },
            start: { $min: '$date' },
            messages: { $addToSet: {
                $format: [
                    '{0}:: {1}',                     // string template
                    { $format: ['$date', 'HH:mm'] }, // replacement for {0}
                    '$content'                       // replacement for {1}
                ]
            } }
        }
    },
    {
        $project: { // cleanup groups infos
            level: '$_id',
            count: 1,
            messages: 1,
            startedAt: { $format: ['$start', 'LT']}
        }
    }
]);
// => [{
//     level: "error",
//     count: 2,
//     messages: [
//         "03:00:: throw an error during date conversion...",
//         "03:30:: throw a new error during date conversion..."
//     ],
//     startedAt: "3:00 AM"
// }]
```
[jsFiddle](http://plnkr.co/edit/k6SUjZLB2doM9WcghDA8?p=preview)

__Note:__ works on nested objects too.

## Installation and usage
1. Choose your preferred method of installation:
 - Through bower: `bower install --save underscore.aggregate`
 - Download from Github: [underscore.aggregate.min.js](https://raw.github.com/nicolaspanel/underscore.aggregate/underscore.aggregate.min.js)

2. Include [underscore](http://underscorejs.org/), [moment](http://momentjs.com/) and [underscore.aggregate](#) in your application:
```html
<script src="components/moment/moment.js"></script>
<script src="components/underscore/underscore.js"></script>
<script src="components/underscore.aggregate/underscore.aggregate.min.js"></script> 
```

## Reference
_See [Quick Reference](#quick-reference) for a more compact overview._

### Stages
The pipeline is an array of stages (i.e. sequence) that aim to transform your collection in a very predictable way.

Usage: 
```javascript
_(collection).aggregate( [ { <stage1> }, { <stage2> }, ... ] );
```

__Note:__
- Collection items pass through the stages in sequence.<commit_msg>update link to minified version<commit_after># Underscore.aggregate

The aggregation pipeline is a framework for data aggregation that make your code easier to read and understand. 

Collection's items (objects) enter a multi-stage pipeline that transforms them and the collection itself into an aggregated results.

__Basic example:__
```js
_([
    { level: 'warn',  date: '1999-12-31 23:59', content: 'dates may be a problem...' },
    { level: 'debug', date: '2000-01-01 00:00', content: 'everything seems to be fine...' },
    { level: 'info',  date: '2000-01-01 01:00', content: 'current date is...' },
    { level: 'warn',  date: '2000-01-01 02:00', content: 'possible memory leak...' },
    { level: 'error', date: '2000-01-01 03:00', content: 'throw an error during date conversion...' },
    { level: 'error', date: '2000-01-01 03:30', content: 'throw a new error during date conversion...' },
    { level: 'info',  date: '2000-01-01 04:00', content: 'everything seems ok now...' }
]).aggregate([
    {  
        $project: { // transform items for the next stage
            level: 1, // <=> level : '$level'
            date: { $parse: '$date' }, // convert string to date
            content: 1
        }
    },
    {
        $match: { // filter items that match following conditions
            level: { $in: ['warn', 'error'] },
            date: {
                $gte: moment('2000-01-01 00:00'),
                $lt: moment('2000-01-02 00:00')
            },
            content: { $regex: /.*date.*/} // filter items whose content contains the word "date"
        }
    },
    {
        $group: { // group remaining items by level and compute aggregated infos
            _id: '$level',
            count: { $sum: 1 },
            start: { $min: '$date' },
            messages: { $addToSet: {
                $format: [
                    '{0}:: {1}',                     // string template
                    { $format: ['$date', 'HH:mm'] }, // replacement for {0}
                    '$content'                       // replacement for {1}
                ]
            } }
        }
    },
    {
        $project: { // cleanup groups infos
            level: '$_id',
            count: 1,
            messages: 1,
            startedAt: { $format: ['$start', 'LT']}
        }
    }
]);
// => [{
//     level: "error",
//     count: 2,
//     messages: [
//         "03:00:: throw an error during date conversion...",
//         "03:30:: throw a new error during date conversion..."
//     ],
//     startedAt: "3:00 AM"
// }]
```
[jsFiddle](http://plnkr.co/edit/k6SUjZLB2doM9WcghDA8?p=preview)

__Note:__ works on nested objects too.

## Installation and usage
1. Choose your preferred method of installation:
 - Through bower: `bower install --save underscore.aggregate`
 - Download from Github: [underscore.aggregate.min.js](https://cdn.rawgit.com/nicolaspanel/underscore.aggregate/master/underscore.aggregate.min.js)

2. Include [underscore](http://underscorejs.org/), [moment](http://momentjs.com/) and [underscore.aggregate](#) in your application:
```html
<script src="components/moment/moment.js"></script>
<script src="components/underscore/underscore.js"></script>
<script src="components/underscore.aggregate/underscore.aggregate.min.js"></script> 
```

## Reference
_See [Quick Reference](#quick-reference) for a more compact overview._

### Stages
The pipeline is an array of stages (i.e. sequence) that aim to transform your collection in a very predictable way.

Usage: 
```javascript
_(collection).aggregate( [ { <stage1> }, { <stage2> }, ... ] );
```

__Note:__
- Collection items pass through the stages in sequence.
{'fraction_non_alphanumeric': 0.1601482091395636, 'fraction_numerical': 0.03760120763002607, 'mean_word_length': 2.2681614349775785, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 23, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5227194', 'n_tokens_mistral': 2383, 'n_tokens_neox': 2121, 'n_words': 653}