<commit_before># DG Toolkit
## Prerequisites
In order to be able to work efficiently you should familiarize yourself with the following technologies:

1. EcmaScript6 aka ES6 aka JS2015 aka Harmony.
The new version of JavaScript. Any valid JS is valid ES6, but ES6 brings to the table some new features
 so it won't hurt to [familiarize](http://es6-features.org/) yourself with it.
 _Note: some of these features(ex. constants) might require a polyfill in order to work in IE9._
 _For a quick and dirty solution try [Babel polyfill](https://babeljs.io/docs/usage/polyfill/)_
2. [React](http://facebook.github.io/react/)
3. [ImmutableJS](https://facebook.github.io/immutable-js/) immutable data structures (not depending on it explicitly,
but NuclearJS does use it for store state)
4. The [Flux](http://facebook.github.io/flux/) architecture. Mainly its [NuclearJS](https://optimizely.github.io/nuclear-js/)
implementation.
5. [Jest](https://facebook.github.io/jest/) testing framework.
6. [Webpack](https://webpack.github.io/)
There's no need to dive deep into it, you'll mainly need to know [how to use loaders](http://webpack.github.io/docs/using-loaders.html)
and [polyfills](http://mts.io/2015/04/08/webpack-shims-polyfills/)

## Usage
Get Node.js

Install webpack globally

    npm install -g webpack

Install module's dependencies

    npm install

Run

    npm start

Navigate to _http://localhost:3000/dev.html_ The browser will be automatically updated
 whenever you save some changes.

To get the project ready for production, run just

    webpack

That will minify and optimize the scripts, use _index.html_ for production.

### Debugging
Add the "debug" key to the local storage to get debug messages
```js
localStorage.debug = "*"
```
### Tests
Run

    npm test

### Pure components
Since we're using immutable data structures, we can reason about whether a component needs to be updated or not
simply by comparing shallowly its previous set of props with current ones. To make use of that, either mix
[PureRenderMixin](https://facebook.github.io/react/docs/pure-render-mixin.html) into your components, or subclass
_components/pure-render-component_<commit_msg>Updated readme to advise building with `npm build` instead of running `webpack` directly. Removed instruction to install webpack globally<commit_after># DG Toolkit
## Prerequisites
In order to be able to work efficiently you should familiarize yourself with the following technologies:

1. EcmaScript6 aka ES6 aka JS2015 aka Harmony.
The new version of JavaScript. Any valid JS is valid ES6, but ES6 brings to the table some new features
 so it won't hurt to [familiarize](http://es6-features.org/) yourself with it.
 _Note: some of these features(ex. constants) might require a polyfill in order to work in IE9._
 _For a quick and dirty solution try [Babel polyfill](https://babeljs.io/docs/usage/polyfill/)_
2. [React](http://facebook.github.io/react/)
3. [ImmutableJS](https://facebook.github.io/immutable-js/) immutable data structures (not depending on it explicitly,
but NuclearJS does use it for store state)
4. The [Flux](http://facebook.github.io/flux/) architecture. Mainly its [NuclearJS](https://optimizely.github.io/nuclear-js/)
implementation.
5. [Jest](https://facebook.github.io/jest/) testing framework.
6. [Webpack](https://webpack.github.io/)
There's no need to dive deep into it, you'll mainly need to know [how to use loaders](http://webpack.github.io/docs/using-loaders.html)
and [polyfills](http://mts.io/2015/04/08/webpack-shims-polyfills/)

## Usage
Get Node.js

Install module's dependencies

    npm install

Run

    npm start

Navigate to _http://localhost:3000/dev.html_ The browser will be automatically updated
 whenever you save some changes.

To get the project ready for production, run

    npm run build

That will minify and optimize the scripts, use _index.html_ for production.

### Debugging
Add the "debug" key to the local storage to get debug messages
```js
localStorage.debug = "*"
```
### Tests
Run

    npm test

### Pure components
Since we're using immutable data structures, we can reason about whether a component needs to be updated or not
simply by comparing shallowly its previous set of props with current ones. To make use of that, either mix
[PureRenderMixin](https://facebook.github.io/react/docs/pure-render-mixin.html) into your components, or subclass
_components/pure-render-component_
{'fraction_non_alphanumeric': 0.09586253673976938, 'fraction_numerical': 0.01266108975808275, 'mean_word_length': 4.414932680538556, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1010856', 'n_tokens_mistral': 1335, 'n_tokens_neox': 1237, 'n_words': 519}