<commit_before># Lede
[![Build Status](https://travis-ci.org/tbtimes/lede.svg?branch=master)](https://travis-ci.org/tbtimes/lede) [![Coverage Status](https://coveralls.io/repos/github/tbtimes/lede/badge.svg?branch=master)](https://coveralls.io/github/tbtimes/lede?branch=master)

Lede is a cross-platform build tool designed to make it simple for news organizations to build pretty story pages. Lede aims to be flexible enough that an experienced designer/developer can quickly throw together a non-traditional story page (like a media slider) but simple enough that non-developers (reporters and web editors) can build story pages from predefined pieces of functionality called bits and blocks. If your newsroom has developers, they can create new bits and blocks for use in any project.

For an easy-to-use command line tool, check out [lede-cli](http://github.com/tbtimes/lede-cli).

__This project is under active development and is, at the moment, incomplete. There may yet be breaking changes before the 1.0.0 release. You can see the design document [here](https://docs.google.com/document/d/1JWlMWI_K7AmLP4jQMzumpTnsQtkIrpMrVWLloAv-M20/edit?usp=sharing).__

## Installation
1. Install [nvm](https://github.com/creationix/nvm) [\[windows\]](https://github.com/coreybutler/nvm-windows) and the latest versions of node (v6.3.x) and npm (v3.x).
2. Create a [GitHub auth token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) and add the environment variable `GH_TOKEN=<your token>`.
3. Ask <NAME> for the Google APIs auth token and set it to the environment variable `GAPI_KEY=<google auth token>`.
4. Install the cli with `npm install -g lede-cli`.

### Technical overview
There are four key concepts to understanding how lede works:
* The Dependency Assembler
* The Cache Builder
* Compilers
* Deployers

There's also a [Lede](./src/lede/Lede.ts) class which simply provides a wrapper around the four components with methods for doing common tasks.

#### Dependency Assembler
Lede has a concept of project inheritance which allows the user to inherit components from other projects. The [Dependency Assembler](./src/lede/DependencyAssembler.ts) is in charge of managing the inheritance chain. It returns a promise wrapped [Project Report](./src/interfaces/ProjectReport.ts) which is the central data structure in a lede project. 

The Project Report specifies the working directory of the current project, an array of every project the current project inherits from, a context object used to build the html, and a few various arrays that specify paths to resources which should be included on the page.

#### Cache Builder
The [Cache Builder](./src/lede/CacheBuilder.ts) takes a Project Report and caches all of the dependencies' resources in the current project's working directory. The cache structure gives lede it's "magic" where projects can import scripts, styles, and html templates from other projects. Compile a lede project and look inside `.ledeCache` to see how it's laid out.

#### Compilers
The compilers are the workhorse of a lede project. Each lede project has three compilers, an html compiler, a css compiler, and a js compiler. 

The html compiler is the root compiler with a `compile` method that takes a Project Report and an object containing the css and js compilers. The `compile` method returns a promise wrapped [Compiled Page](./src/interfaces/CompiledPage.ts). 

The html compiler uses the project report to create an array of bits that are included in the project; it then passes the project report and that array of used bits to the other two compilers' `compile` method. Each of those methods return a promise wrapped object with two properties, bits and globals, each containing a string of the compiled styles/scripts.

Lede's default compilers are Nunjucks for html, Sass for css, and Es6 for javascript but you can easily swap them out for custom built compilers that conform to the spec, keeping in mind that one set of compilers must work for every project in the inheritance chain (ie: no ElmCompiler for a project that inherits ES6 scripts.)

Compilers are specified in the current project inside `compilerConfig.js`. At the most basic level, compiler config should export an object with `compilers.html`, `compilers.css`, and `compilers.js` properties which implement the html, css, and js compiler spec described above. TODO: make these interfaces and link to them here. 
<commit_msg>updated install process<commit_after># Lede
[![Build Status](https://travis-ci.org/tbtimes/lede.svg?branch=master)](https://travis-ci.org/tbtimes/lede) [![Coverage Status](https://coveralls.io/repos/github/tbtimes/lede/badge.svg?branch=master)](https://coveralls.io/github/tbtimes/lede?branch=master)

Lede is a cross-platform build tool designed to make it simple for news organizations to build pretty story pages. Lede aims to be flexible enough that an experienced designer/developer can quickly throw together a non-traditional story page (like a media slider) but simple enough that non-developers (reporters and web editors) can build story pages from predefined pieces of functionality called bits and blocks. If your newsroom has developers, they can create new bits and blocks for use in any project.

For an easy-to-use command line tool, check out [lede-cli](http://github.com/tbtimes/lede-cli).

__This project is under active development and is, at the moment, incomplete. There may yet be breaking changes before the 1.0.0 release. You can see the design document [here](https://docs.google.com/document/d/1JWlMWI_K7AmLP4jQMzumpTnsQtkIrpMrVWLloAv-M20/edit?usp=sharing).__

## Installation
1. Install [nvm](https://github.com/creationix/nvm) [\[windows\]](https://github.com/coreybutler/nvm-windows) and the latest versions of node (v6.3.x) and npm (v3.x).
2. Create a [GitHub auth token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) and add the environment variable `GH_TOKEN=<your token>`.
3. Ask <NAME> for the Google APIs auth token and set it to the environment variable `GAPI_KEY=<google auth token>`.
4. Install the cli with `npm install -g lede-cli`.
5. Install bunyan for pretty printing lede logs (`npm install -g bunyan`).

### Technical overview
There are four key concepts to understanding how lede works:
* The Dependency Assembler
* The Cache Builder
* Compilers
* Deployers

There's also a [Lede](./src/lede/Lede.ts) class which simply provides a wrapper around the four components with methods for doing common tasks.

#### Dependency Assembler
Lede has a concept of project inheritance which allows the user to inherit components from other projects. The [Dependency Assembler](./src/lede/DependencyAssembler.ts) is in charge of managing the inheritance chain. It returns a promise wrapped [Project Report](./src/interfaces/ProjectReport.ts) which is the central data structure in a lede project. 

The Project Report specifies the working directory of the current project, an array of every project the current project inherits from, a context object used to build the html, and a few various arrays that specify paths to resources which should be included on the page.

#### Cache Builder
The [Cache Builder](./src/lede/CacheBuilder.ts) takes a Project Report and caches all of the dependencies' resources in the current project's working directory. The cache structure gives lede it's "magic" where projects can import scripts, styles, and html templates from other projects. Compile a lede project and look inside `.ledeCache` to see how it's laid out.

#### Compilers
The compilers are the workhorse of a lede project. Each lede project has three compilers, an html compiler, a css compiler, and a js compiler. 

The html compiler is the root compiler with a `compile` method that takes a Project Report and an object containing the css and js compilers. The `compile` method returns a promise wrapped [Compiled Page](./src/interfaces/CompiledPage.ts). 

The html compiler uses the project report to create an array of bits that are included in the project; it then passes the project report and that array of used bits to the other two compilers' `compile` method. Each of those methods return a promise wrapped object with two properties, bits and globals, each containing a string of the compiled styles/scripts.

Lede's default compilers are Nunjucks for html, Sass for css, and Es6 for javascript but you can easily swap them out for custom built compilers that conform to the spec, keeping in mind that one set of compilers must work for every project in the inheritance chain (ie: no ElmCompiler for a project that inherits ES6 scripts.)

Compilers are specified in the current project inside `compilerConfig.js`. At the most basic level, compiler config should export an object with `compilers.html`, `compilers.css`, and `compilers.js` properties which implement the html, css, and js compiler spec described above. TODO: make these interfaces and link to them here. 

{'fraction_non_alphanumeric': 0.07287853577371048, 'fraction_numerical': 0.003882418191902385, 'mean_word_length': 4.809278350515464, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 16, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4274888', 'n_tokens_mistral': 2416, 'n_tokens_neox': 2330, 'n_words': 1198}