<commit_before># Build setup

#### Windows note: use an Ubuntu VM

Many Windows developers have found that setting up an Ubuntu virtual machine is less painful than getting Ruby and other prerequisites running on Windows.

* Option A: Use [VMWare Player](https://my.vmware.com/web/vmware/free#desktop_end_user_computing/vmware_player/4_0) and an [Ubuntu 14.04 iso image](http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-desktop-amd64.iso)
* Option B: Use vagrant ([install](https://docs.vagrantup.com/v2/installation/)):
  1. First clone the code.org git repo to get the provided Vagrantfile (you will be able to skip step 1 of the common setup instructions): `git clone https://github.com/code-dot-org/code-dot-org.git`
  1. `cd code-dot-org`
  1. `vagrant up`
  1. `vagrant ssh`
  1. Goto step 2 of the common setup instructions
* Option C: Use AWS EC2: [launch Ubuntu 14.04 AMI](https://console.aws.amazon.com/ec2/home?region=ap-northeast-1#launchAmi=ami-d9fdddd8)

## Install OS-specific prerequisites

### OS X Mavericks / Yosemite

1. Install Homebrew: `ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"`
1. `brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb enscript gs mysql imagemagick rbenv ruby-build coreutils`
  1. If it complains about an old version of `<package>`, run `brew unlink <package>` and run `brew install <package>` again
1. Set up MySQL
  1. Have launchd start mysql at login: `ln -sfv /usr/local/opt/mysql/*.plist ~/Library/LaunchAgents`
  1. Start mysql now: `launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist`
1. Set up [RBENV](https://github.com/sstephenson/rbenv#homebrew-on-mac-os-x) and Ruby 2.0
  1. Add `if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi` to `~/.profile`
  1. source `~/.profile`
  1. rbenv install 2.0.0-p451
  1. rbenv global 2.0.0-p451
  1. rbenv rehash

### Ubuntu 14.04

1. `sudo aptitude update`
1. `sudo aptitude upgrade`
1. `sudo aptitude install -y git mysql-server mysql-client libmysqlclient-dev libxslt1-dev libssl-dev zlib1g-dev imagemagick libmagickcore-dev libmagickwand-dev nodejs npm openjdk-7-jre-headless libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev curl pdftk ruby2.0 ruby2.0-dev`
  * **Hit enter and select default options for any configuration popups**
1. Either setup RBENV or configure your default ruby and gem version to 2.0
  1. Option A - RBENV: ([instructions](https://github.com/sstephenson/rbenv#installation))
  1. Option B - Symlinks:
    1. Ruby: `sudo ln -sf /usr/bin/ruby2.0 /usr/bin/ruby`
    1. Gem: `sudo ln -sf /usr/bin/gem2.0 /usr/bin/gem`

## Common setup

1. `git clone https://github.com/code-dot-org/code-dot-org.git`
1. `gem install bundler`
1. `rbenv rehash` (if using rbenv)
1. `cd code-dot-org/aws`
1. `bundle install`
1. `cd ..`
1. `rake install`

## Organizational Structure

Our code is segmented into four parts:

* Blockly Core is the visual programming language platform used for the interactive tutorials.
* Blockly includes apps—blockly puzzles built based on Blockly Core.
  * [Hour of Code](http://studio.code.org/hoc/1)
* Dashboard, is the tutorial platform which organizes blockly levels into tutorials.
  * [Code Studio](http://studio.code.org)
* Pegasus is the main site which also includes the teacher dashboard (support for teachers to track student progress).
  * [code.org](http://code.org)
  * [csedweek.org](http://csedweek.org)
  * [Teacher Dashboard](http://code.org/teacher-dashboard)
<commit_msg>Readme setup changes<commit_after># Build setup

#### Windows note: use an Ubuntu VM

Many Windows developers have found that setting up an Ubuntu virtual machine is less painful than getting Ruby and other prerequisites running on Windows.

* Option A: Use [VMWare Player](https://my.vmware.com/web/vmware/free#desktop_end_user_computing/vmware_player/4_0) and an [Ubuntu 14.04 iso image](http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-desktop-amd64.iso)
* Option B: Use vagrant ([install](https://docs.vagrantup.com/v2/installation/)):
  1. First clone the code.org git repo to get the provided Vagrantfile (you will be able to skip step 1 of the common setup instructions): `git clone https://github.com/code-dot-org/code-dot-org.git`
  1. `cd code-dot-org`
  1. `vagrant up`
  1. `vagrant ssh`
  1. Goto step 2 of the common setup instructions
* Option C: Use AWS EC2: [launch Ubuntu 14.04 AMI](https://console.aws.amazon.com/ec2/home?region=ap-northeast-1#launchAmi=ami-d9fdddd8)

## Install OS-specific prerequisites

### OS X Mavericks / Yosemite

1. Install Homebrew: `ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"`
1. `brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb enscript gs mysql imagemagick rbenv ruby-build coreutils`
  1. If it complains about an old version of `<package>`, run `brew unlink <package>` and run `brew install <package>` again
1. Set up MySQL
  1. Have launchd start mysql at login: `ln -sfv /usr/local/opt/mysql/*.plist ~/Library/LaunchAgents`
  1. Start mysql now: `launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist`
1. Set up [RBENV](https://github.com/sstephenson/rbenv#homebrew-on-mac-os-x) and Ruby 2.0
  1. Add `if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi` to `~/.profile`
  1. source `~/.profile`
  1. `rbenv install 2.0.0-p451`
  1. `rbenv global 2.0.0-p451`
  1. `rbenv rehash`

### Ubuntu 14.04

1. `curl -sL https://deb.nodesource.com/setup | sudo bash -`
1. `sudo aptitude update`
1. `sudo aptitude upgrade`
1. `sudo aptitude install -y git mysql-server mysql-client libmysqlclient-dev libxslt1-dev libssl-dev zlib1g-dev imagemagick libmagickcore-dev libmagickwand-dev nodejs openjdk-7-jre-headless libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev curl pdftk ruby2.0 ruby2.0-dev`
  * **Hit enter and select default options for any configuration popups**
1. Either setup RBENV or configure your default ruby and gem version to 2.0
  1. Option A - RBENV: ([instructions](https://github.com/sstephenson/rbenv#installation))
    1. Install RBENV and ruby-build
    1. `rbenv install 2.0.0-p451`
    1. `rbenv global 2.0.0-p451`
    1. `rbenv rehash`
  1. Option B - Symlinks:
    1. Ruby: `sudo ln -sf /usr/bin/ruby2.0 /usr/bin/ruby`
    1. Gem: `sudo ln -sf /usr/bin/gem2.0 /usr/bin/gem`
    1. `sudo chown \`whoami\` /usr/bin/gem/`

## Common setup

1. `git clone https://github.com/code-dot-org/code-dot-org.git`
1. `gem install bundler`
1. `rbenv rehash` (if using rbenv)
1. `cd code-dot-org/aws`
1. `bundle install`
1. `cd ..`
1. `sudo chown \`whoami\` /usr/bin/gem/ 
1. `rake install`

## Organizational Structure

Our code is segmented into four parts:

* Blockly Core is the visual programming language platform used for the interactive tutorials.
* Blockly includes apps—blockly puzzles built based on Blockly Core.
  * [Hour of Code](http://studio.code.org/hoc/1)
* Dashboard, is the tutorial platform which organizes blockly levels into tutorials.
  * [Code Studio](http://studio.code.org)
* Pegasus is the main site which also includes the teacher dashboard (support for teachers to track student progress).
  * [code.org](http://code.org)
  * [csedweek.org](http://csedweek.org)
  * [Teacher Dashboard](http://code.org/teacher-dashboard)

{'fraction_non_alphanumeric': 0.12853049529267294, 'fraction_numerical': 0.029335516441533635, 'mean_word_length': 3.8607427055702916, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 19, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4896510', 'n_tokens_mistral': 2738, 'n_tokens_neox': 2542, 'n_words': 803}