<commit_before>This document describes how to set up the MITx development environment
for both Linux (Ubuntu) and MacOS (OSX Lion). 

There is also a script "create-dev-env.sh" that automates these steps.

1) Make an mitx_all directory and clone the repos
  (download and install git and mercurial if you don't have them already)

    mkdir ~/mitx_all
    cd ~/mitx_all
    git clone git<EMAIL>:MITx/mitx.git
    git clone git@github.com:MITx/askbot-devel 
    hg clone ssh://hg-content@gp.<EMAIL>.<EMAIL>.<EMAIL>/data 

2) Install OSX dependencies (Mac users only)

    a) Install the brew utility if necessary
    /usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/master/Library/Contributions/install_homebrew.rb)"

    b) Install the brew package list
    cat ~/mitx_all/mitx/brew-formulas.txt | xargs brew install

    c) Install python pip if necessary
    sudo easy_install pip  

    d) Install python virtualenv if necessary
    sudo pip install virtualenv virtualenvwrapper

    e) Install coffee script
    curl http://npmjs.org/install.sh | sh
    npm install -g coffee-script

3) Install Ubuntu dependencies (Linux users only)
    
    sudo apt-get install curl python-virtualenv build-essential python-dev gfortran liblapack-dev libfreetype6-dev libpng12-dev libxml2-dev libxslt-dev yui-compressor coffeescript


4) Install rvm, ruby, and libraries

    echo "export rvm_path=$HOME/mitx_all/ruby" > $HOME/.rvmrc
    curl -sL get.rvm.io | bash -s stable
    source ~/mitx_all/ruby/scripts/rvm
    rvm install 1.9.3
    gem install bundler
    cd ~/mitx_all/mitx
    bundle install
   
5) Install python libraries
    
    source ~/mitx_all/python/bin/activate
    cd ~/mitx_all
    pip install -r askbot-devel/askbot_requirements.txt 
    pip install -r askbot-devel/askbot_requirements_dev.txt 
    pip install -r mitx/pre-requirements.txt 
    pip install -r mitx/requirements.txt 

6) Create log and db dirs

    mkdir ~/mitx_all/log
    mkdir ~/mitx_all/db

7) Start the dev server

   To start using Django you will need
   to activate the local Python and Ruby
   environment:

        $ source ~/mitx_all/ruby/scripts/rvm
        $ source ~/mitx_all/python/bin/activate
  
   To initialize and start a local instance of Django:
        
        $ cd ~/mitx_all/mitx
        $ django-admin.py syncdb --settings=envs.dev --pythonpath=.
        $ django-admin.py migrate --settings=envs.dev --pythonpath=.
        $ django-admin.py runserver --settings=envs.dev --pythonpath=.   
 
<commit_msg>Meaningless commit to force build<commit_after>This document describes how to set up the MITx development environment
for both Linux (Ubuntu) and MacOS (OSX Lion). 

There is also a script "create-dev-env.sh" that automates these steps.

1) Make an mitx_all directory and clone the repos
  (download and install git and mercurial if you don't have them already)

    mkdir ~/mitx_all
    cd ~/mitx_all
    git clone git<EMAIL>:MITx/mitx.git
    git clone git@github.com:MITx/askbot-devel 
    hg clone ssh://hg-content@gp.mitx.mit.edu/data 

2) Install OSX dependencies (Mac users only)

    a) Install the brew utility if necessary
    /usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/master/Library/Contributions/install_homebrew.rb)"

    b) Install the brew package list
    cat ~/mitx_all/mitx/brew-formulas.txt | xargs brew install

    c) Install python pip if necessary
    sudo easy_install pip  

    d) Install python virtualenv if necessary
    sudo pip install virtualenv virtualenvwrapper

    e) Install coffee script
    curl http://npmjs.org/install.sh | sh
    npm install -g coffee-script

3) Install Ubuntu dependencies (Linux users only)
    
    sudo apt-get install curl python-virtualenv build-essential python-dev gfortran liblapack-dev libfreetype6-dev libpng12-dev libxml2-dev libxslt-dev yui-compressor coffeescript


4) Install rvm, ruby, and libraries

    echo "export rvm_path=$HOME/mitx_all/ruby" > $HOME/.rvmrc
    curl -sL get.rvm.io | bash -s stable
    source ~/mitx_all/ruby/scripts/rvm
    rvm install 1.9.3
    gem install bundler
    cd ~/mitx_all/mitx
    bundle install
   
5) Install python libraries
    
    source ~/mitx_all/python/bin/activate
    cd ~/mitx_all
    pip install -r askbot-devel/askbot_requirements.txt 
    pip install -r askbot-devel/askbot_requirements_dev.txt 
    pip install -r mitx/pre-requirements.txt 
    pip install -r mitx/requirements.txt 

6) Create log and db dirs

    mkdir ~/mitx_all/log
    mkdir ~/mitx_all/db

7) Start the dev server

   To start using Django you will need
   to activate the local Python and Ruby
   environment:

        $ source ~/mitx_all/ruby/scripts/rvm
        $ source ~/mitx_all/python/bin/activate
  
   To initialize and start a local instance of Django:
        
        $ cd ~/mitx_all/mitx
        $ django-admin.py syncdb --settings=envs.dev --pythonpath=.
        $ django-admin.py migrate --settings=envs.dev --pythonpath=.
        $ django-admin.py runserver --settings=envs.dev --pythonpath=.   

{'fraction_non_alphanumeric': 0.08413983804068734, 'fraction_numerical': 0.005530317993284614, 'mean_word_length': 2.922540666150271, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 10, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 3, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '166738', 'n_tokens_mistral': 1713, 'n_tokens_neox': 1593, 'n_words': 514}