<commit_before>## Build on Linux: ##

1. If installing in a virtual machine:
    1. Bear in mind that Torch doesn't support Debian at the moment. Ubuntu (latest) is recommended.
    2. Choose the 32- or 64-bit OS version that you want to target. If you don't know, use 64-bit.
    3. Provide at least 4GB of memory and 20GB of virtual disk space.

2. Install dependencies.
    1. On Ubuntu 15.10:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libboost-all-dev libpython2.7-dev lua5.1 liblua5.1-0-dev openjdk-8-jdk swig xsdcxx libxerces-c-dev doxygen xsltproc ffmpeg`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/`  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)
         
    2. On Ubuntu 14.04:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libboost-all-dev libpython2.7-dev lua5.1 liblua5.1-0-dev openjdk-7-jdk swig libxerces-c-dev doxygen xsltproc libav-tools`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/`  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)
         
    3. On Debian 8:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libboost-all-dev libpython2.7-dev lua5.1 liblua5.1-0-dev openjdk-7-jdk swig xsdcxx libxerces-c-dev doxygen xsltproc libav-tools`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/`  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)
         
    4. On Debian 7:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libbz2-dev python2.7-dev lua5.1 liblua5.1-0-dev openjdk-7-jdk swig libxerces-c-dev doxygen xsltproc ffmpeg`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/`  
         
         Also, remove Java 6 and make sure that `java -version` returns the right version (1.7).  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)

    5. On Fedora 23:  

        `su -c 'dnf install http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm'` (for ffmpeg)  

        `sudo dnf install git cmake cmake-gui boost-devel python-devel java-1.8.0-openjdk-devel swig xsd xerces-c-devel doxygen libxslt ffmpeg gcc-c++ mono-devel compat-lua compat-lua-devel lua-socket-compat`

4. Install Torch: (if supported by your platform)
    1. Follow the instructions at http://torch.ch/docs/getting-started.html
    2. Test: `th`

5. Install Mono: (if not Fedora)
    1. The Mono Project has an excellent [Getting Started](http://www.mono-project.com/docs/) guide, please read it.
    2. For the impatient, Linux details are [here](http://www.mono-project.com/docs/getting-started/install/linux/)
    <commit_msg>Minor: added python-tk package to build instructions, since needed for integration tests.<commit_after>## Build on Linux: ##

1. If installing in a virtual machine:
    1. Bear in mind that Torch doesn't support Debian at the moment. Ubuntu (latest) is recommended.
    2. Choose the 32- or 64-bit OS version that you want to target. If you don't know, use 64-bit.
    3. Provide at least 4GB of memory and 20GB of virtual disk space.

2. Install dependencies.
    1. On Ubuntu 15.10:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libboost-all-dev libpython2.7-dev lua5.1 liblua5.1-0-dev openjdk-8-jdk swig xsdcxx libxerces-c-dev doxygen xsltproc ffmpeg python-tk`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/`  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)
         
    2. On Ubuntu 14.04:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libboost-all-dev libpython2.7-dev lua5.1 liblua5.1-0-dev openjdk-7-jdk swig libxerces-c-dev doxygen xsltproc libav-tools python-tk`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/`  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)
         
    3. On Debian 8:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libboost-all-dev libpython2.7-dev lua5.1 liblua5.1-0-dev openjdk-7-jdk swig xsdcxx libxerces-c-dev doxygen xsltproc libav-tools python-tk`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/`  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)
         
    4. On Debian 7:  
    
         `sudo apt-get install build-essential git cmake cmake-qt-gui libbz2-dev python2.7-dev lua5.1 liblua5.1-0-dev openjdk-7-jdk swig libxerces-c-dev doxygen xsltproc ffmpeg python-tk`  
         
         `export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/`  
         
         Also, remove Java 6 and make sure that `java -version` returns the right version (1.7).  
         
         `sudo update-ca-certificates -f` (http://stackoverflow.com/a/29313285/126823)

    5. On Fedora 23:  

        `su -c 'dnf install http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm'` (for ffmpeg)  

        `sudo dnf install git cmake cmake-gui boost-devel python-devel java-1.8.0-openjdk-devel swig xsd xerces-c-devel doxygen libxslt ffmpeg gcc-c++ mono-devel compat-lua compat-lua-devel lua-socket-compat tkinter`

4. Install Torch: (if supported by your platform)
    1. Follow the instructions at http://torch.ch/docs/getting-started.html
    2. Test: `th`

5. Install Mono: (if not Fedora)
    1. The Mono Project has an excellent [Getting Started](http://www.mono-project.com/docs/) guide, please read it.
    2. For the impatient, Linux details are [here](http://www.mono-project.com/docs/getting-started/install/linux/)
    
{'fraction_non_alphanumeric': 0.11234850966262692, 'fraction_numerical': 0.047821814608581724, 'mean_word_length': 2.491709548313322, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '305485', 'n_tokens_mistral': 2398, 'n_tokens_neox': 2143, 'n_words': 550}