<commit_before># Sprockets::Commoner

Uses Node directly to run Babel instead of ExecJS. This gives the benefit of being able to use any Babel plugin and configuration, without depending on them being vendored into the gem.

## Methodology

Commoner registers a postprocessor that takes any `application/javascript` file and passes it through Babel.

Commoner will work its magic on any JavaScript file that has a `.babelrc` somewhere in its tree.

It works by spawning a node process to do the actual translation. It then sends messages back and forth to this process with code. The reason it uses a Node process instead of ExecJS is because ExecJS has a lot less flexibility in allowing a JavaScript backend to specify its environment, so they can't do things like `require` outside packages. This means that any gem that uses it has to vendor in all the JavaScript dependencies, which is what `sprockets-es6` does.

### `require` support

`Sprockets::Commoner` supports `require()` by replacing `require` function calls with variable references. It does this by assigning every file a unique identifier based on their filename. For example, if the file you're referencing is located at `<root>/node_modules/package/index.js` it will get the name `__commoner_module__node_modules$package$index_js`. Any file that then `require`s this specific file will have that `require` call replaced with the identifier.

If the resolved file is a CoffeeScript file however, it needs to do another trick. The resolving step will execute a RegExp against the file to find any global variable definitions inside the file, and will replace the `require` call with a global variable reference.

The Babel plugin also communicates back any files that were required to make sure they are included by Sprockets. `Sprockets::Commoner` depends on the topological ordering that Sprockets does to make sure any module that is needed is instantiated before it is used.

## Steps

`Sprockets::Commoner` works in a couple of steps. When the processor starts up it spins off a Node.js process in the root directory of the Sprockets environment (which is `Rails.root` in a Rails app). This is also where you should install `babel-core` to make sure the process has access to it. `Sprockets::Commoner` doesn't include Babel, so you need to install this yourself.

### Step 1

First the file is passed through any Babel plugins that are defined in your `.babelrc`. You shouldn't include `babel-plugin-transform-es2015-modules-commonjs` because the  internal Babel plugin in `Sprockets::Commoner` does this already.<commit_msg>Add setup instructions to README<commit_after># Sprockets::Commoner

Uses Node directly to run Babel instead of ExecJS. This gives the benefit of being able to use any Babel plugin and configuration, without depending on them being vendored into the gem.

## Setup

1. `rails new project; cd project`
2. Add sprockets-commoner to Gemfile and `bundle install`
3. Add package.json with `babel-core`, `browser-resolve`, `babel-types`, `babel-generator`, `babel-helpers` and any client-side packages you need (can be simplified).
4. Remove `config.assets.debug = true` from `config/environments/development.rb` (can be done automatically)
5. Add `Rails.application.config.assets.paths << Rails.root.join('node_modules')` to `config/initializers/assets.rb` (can be done automatically)
6. Add `.babelrc` with any config to the project root
7. `require/import` your client-side things from `application.js`!


## Methodology

Commoner registers a postprocessor that takes any `application/javascript` file and passes it through Babel.

Commoner will work its magic on any JavaScript file that has a `.babelrc` somewhere in its tree.

It works by spawning a node process to do the actual translation. It then sends messages back and forth to this process with code. The reason it uses a Node process instead of ExecJS is because ExecJS has a lot less flexibility in allowing a JavaScript backend to specify its environment, so they can't do things like `require` outside packages. This means that any gem that uses it has to vendor in all the JavaScript dependencies, which is what `sprockets-es6` does.

### `require` support

`Sprockets::Commoner` supports `require()` by replacing `require` function calls with variable references. It does this by assigning every file a unique identifier based on their filename. For example, if the file you're referencing is located at `<root>/node_modules/package/index.js` it will get the name `__commoner_module__node_modules$package$index_js`. Any file that then `require`s this specific file will have that `require` call replaced with the identifier.

If the resolved file is a CoffeeScript file however, it needs to do another trick. The resolving step will execute a RegExp against the file to find any global variable definitions inside the file, and will replace the `require` call with a global variable reference.

The Babel plugin also communicates back any files that were required to make sure they are included by Sprockets. `Sprockets::Commoner` depends on the topological ordering that Sprockets does to make sure any module that is needed is instantiated before it is used.

## Steps

`Sprockets::Commoner` works in a couple of steps. When the processor starts up it spins off a Node.js process in the root directory of the Sprockets environment (which is `Rails.root` in a Rails app). This is also where you should install `babel-core` to make sure the process has access to it. `Sprockets::Commoner` doesn't include Babel, so you need to install this yourself.

### Step 1

First the file is passed through any Babel plugins that are defined in your `.babelrc`. You shouldn't include `babel-plugin-transform-es2015-modules-commonjs` because the  internal Babel plugin in `Sprockets::Commoner` does this already.
{'fraction_non_alphanumeric': 0.0568259385665529, 'fraction_numerical': 0.003242320819112628, 'mean_word_length': 4.8551448551448555, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2689044', 'n_tokens_mistral': 1501, 'n_tokens_neox': 1443, 'n_words': 849}