<commit_before># Orchestrator for Developers

`orchestrator` is open source and accepts pull requests.

If you would like to build `orchestrator` on your own machine, or eventually submit PRs, follow this guide.

### Requirements

- `orchestrator` is built on Linux and OS/X. I have no hint about MS Windows, and the build is incompatible with Windows.

- `Go 1.7` or higher.

### Clone repo

`orchestrator` is built with `Go`. `Go` is picky about where you place your code (tl;dr: in a well structured path under `$GOPATH`). However the good news is we have the scripting to go around that (no pun intended).

### Easy clone + builds

Clone the repo anywhere on your filesystem via

	git clone git@github.com:github/orchestrator.git
	cd orchestrator

Build `orchestrator` via

	script/build


You will find the binary as `bin/orchestrator`

### Not as easy clone + builds

Why would you want this? Because this will empower you with building `.DEB`, `.rpm` packages for both Linux and OS/X.

- Make sure `GOPATH` is set
- Issue:

	go get github.com/github/orchestrator/...
	cd $GOPATH:/src/github.com/github/orchestrator

- Compile or run via:

	go build -i go/cmd/orchestrator/main.go
	go run go/cmd/orchestrator/main.go

- Create packages via:

	./build.sh

To create packages you will need to have:

 - [fpm](https://github.com/jordansissel/fpm), which assumes you have `ruby` and `ruby-gems`
 - `rpmbuild`
 - `go`, `gofmt` in path
 - `tar`

### DB setup

`orchestrator` requires a MySQL backend to run. This could be installed anywhere. I usually use [mysqlsandbox](http://mysqlsandbox.net/) for local installations. You may choose to just install mysql-server on your dev machine.

Once your backend MySQL setup is complete, issue:

    CREATE DATABASE IF NOT EXISTS orchestrator;
    GRANT ALL PRIVILEGES ON `orchestrator`.* TO 'orchestrator'@'127.0.0.1'
    IDENTIFIED BY 'orch_backend_password';

`orchestrator` uses a configuration file whose search path is either `/etc/orchestrator.conf.json`,  `conf/orchestrator.conf.json` or `orchestrator.conf.json`.
The repository includes a file called `conf/orchestrator-sample.conf.json` with some basic settings. Issue:

	cp conf/orchestrator-sample.conf.json conf/orchestrator.conf.json

The `conf/orchestrator.conf.json` file is not part of the repository and there is in fact a `.gitignore` entry for this file.<commit_msg>ongoing doc updates<commit_after># Orchestrator for Developers

`orchestrator` is open source and accepts pull requests.

If you would like to build `orchestrator` on your own machine, or eventually submit PRs, follow this guide.

### Requirements

- `orchestrator` is built on Linux and OS/X. I have no hint about MS Windows, and the build is incompatible with Windows.

- `Go 1.7` or higher.

### Clone repo

`orchestrator` is built with `Go`. `Go` is picky about where you place your code (tl;dr: in a well structured path under `$GOPATH`). However the good news is we have the scripting to go around that (no pun intended).

### Easy clone + builds

Clone the repo anywhere on your filesystem via

	git clone git@github.com:github/orchestrator.git
	cd orchestrator

Build `orchestrator` via

	script/build


You will find the binary as `bin/orchestrator`

### Not as easy clone + builds

Why would you want this? Because this will empower you with building `.DEB`, `.rpm` packages for both Linux and OS/X.

- Make sure `GOPATH` is set
- Issue:

	  go get github.com/github/orchestrator/...
	  cd $GOPATH:/src/github.com/github/orchestrator

- Compile or run via:

	  go build -i go/cmd/orchestrator/main.go
	  go run go/cmd/orchestrator/main.go

- Create packages via:

	  ./build.sh

To create packages you will need to have:

 - [fpm](https://github.com/jordansissel/fpm), which assumes you have `ruby` and `ruby-gems`
 - `rpmbuild`
 - `go`, `gofmt` in path
 - `tar`

### DB setup

`orchestrator` requires a MySQL backend to run. This could be installed anywhere. I usually use [mysqlsandbox](http://mysqlsandbox.net/) for local installations. You may choose to just install mysql-server on your dev machine.

Once your backend MySQL setup is complete, issue:

    CREATE DATABASE IF NOT EXISTS orchestrator;
    GRANT ALL PRIVILEGES ON `orchestrator`.* TO 'orchestrator'@'127.0.0.1'
    IDENTIFIED BY 'orch_<PASSWORD>';

`orchestrator` uses a configuration file whose search path is either `/etc/orchestrator.conf.json`,  `conf/orchestrator.conf.json` or `orchestrator.conf.json`.
The repository includes a file called `conf/orchestrator-sample.conf.json` with some basic settings. Issue:

	cp conf/orchestrator-sample.conf.json conf/orchestrator.conf.json

The `conf/orchestrator.conf.json` file is not part of the repository and there is in fact a `.gitignore` entry for this file.
{'fraction_non_alphanumeric': 0.09350104821802935, 'fraction_numerical': 0.0033542976939203353, 'mean_word_length': 3.9749739311783108, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3903298', 'n_tokens_mistral': 1530, 'n_tokens_neox': 1510, 'n_words': 603}