<commit_before>OpenGEM System
==============

Dependencies
------------

For the most part the Python packages in here can be installed using `pip`
(python-pip on ubuntu) or your favorite package manager.

On OS X a good way to intall the pieces that are not Python (if you don't
already use MacPorts or Fink) is called homebrew: http://mxcl.github.com/homebrew/

* eventlet
* lxml
* PyYAML
* python-gflags
* RabbitMQ (Installed using brew install rabbitmq)
* SciPy
  * You will need gfortran, for os x that is at: http://r.research.att.com/gfortran-4.2.3.dmg 
  * After that `pip install scipy` should work.
  * If it doesn't and there is an error about ppc64... well it is more complex,
    see Troubleshooting
* Shapely
  * requires geos (c library, also called libgeos)
* Sphinx (for building documentation only)
* Guppy (http://guppy-pe.sourceforge.net)


Running Tests
-------------

To run the python tests use:

::

    python runtests.py

To run the java tests use:

::

    ant


Running Demos
-------------

To run the command-line demo:

::

    ./demo.sh --debug



Tools / Services
----------------

* git
* git-cl (http://neugierig.org/software/git/?r=git-cl)

::
    
    git clone git://neugierig.org/git-cl.git
    ln -s /path/to/git-cl/git-cl /usr/bin/git-cl
    ln -s /path/to/git-cl/upload.py /usr/bin/upload.py

* GitHub (http://github.com/gem/opengem)
* Rietveld (http://gemreview.appspot.com) - requires a google account
* PivotalTracker (http://pivotaltracker.com)

Troubleshooting
---------------

SciPy can be a real pain to install on OS X. These steps should work if the
procedure mentioned above does not.

Make sure you have installed numpy >= 1.4, if you have installed GDAL you 
might have an older version of numpy that may be loaded instead of your new
installation on numpy. To resolve this, change the python module loading order
by editing /Python/2.6/site-packages/dgal.pth
from:
import sys; sys.path.insert(0,'/Library/Frameworks/GDAL.framework/Versions/1.7/
Python/site-packages')
to:
import sys; sys.path.append('/Library/Frameworks/GDAL.framework/Versions/1.7/
Python/site-packages')

::

    svn co http://svn.scipy.org/svn/numpy/trunk numpy 
    svn co http://svn.scipy.org/svn/scipy/trunk scipy
    
    cd numpy

    # edit gnu.py and find the word "ppc64" and remove it from the array
    vim numpy/distutils/fcompiler/gnu.py

    python setup.py build
    sudo python setup.py install<commit_msg>add documentation for GDAL numpy conflict<commit_after>OpenGEM System
==============

Dependencies
------------

For the most part the Python packages in here can be installed using `pip`
(python-pip on ubuntu) or your favorite package manager.

On OS X a good way to intall the pieces that are not Python (if you don't
already use MacPorts or Fink) is called homebrew: http://mxcl.github.com/homebrew/

* eventlet
* lxml
* PyYAML
* python-gflags
* RabbitMQ (Installed using brew install rabbitmq)
* SciPy
  * You will need gfortran, for os x that is at: http://r.research.att.com/gfortran-4.2.3.dmg 
  * After that `pip install scipy` should work.
  * If it doesn't and there is an error about ppc64... well it is more complex,
    see Troubleshooting
* Shapely
  * requires geos (c library, also called libgeos)
* Sphinx (for building documentation only)
* Guppy (http://guppy-pe.sourceforge.net)


Running Tests
-------------

To run the python tests use:

::

    python runtests.py

To run the java tests use:

::

    ant


Running Demos
-------------

To run the command-line demo:

::

    ./demo.sh --debug



Tools / Services
----------------

* git
* git-cl (http://neugierig.org/software/git/?r=git-cl)

::
    
    git clone git://neugierig.org/git-cl.git
    ln -s /path/to/git-cl/git-cl /usr/bin/git-cl
    ln -s /path/to/git-cl/upload.py /usr/bin/upload.py

* GitHub (http://github.com/gem/opengem)
* Rietveld (http://gemreview.appspot.com) - requires a google account
* PivotalTracker (http://pivotaltracker.com)

Troubleshooting
---------------

SciPy can be a real pain to install on OS X. These steps should work if the
procedure mentioned above does not.

Make sure you have installed numpy >= 1.4, if you have installed GDAL you 
might have an older version of numpy that may be loaded by Python before your
new installation on numpy. To resolve this, change the python module loading 
order by editing /Python/2.6/site-packages/dgal.pth
from:
import sys; sys.path.insert(0,'/Library/Frameworks/GDAL.framework/Versions/1.7/
Python/site-packages')
to:
import sys; sys.path.append('/Library/Frameworks/GDAL.framework/Versions/1.7/
Python/site-packages')

::

    svn co http://svn.scipy.org/svn/numpy/trunk numpy 
    svn co http://svn.scipy.org/svn/scipy/trunk scipy
    
    cd numpy

    # edit gnu.py and find the word "ppc64" and remove it from the array
    vim numpy/distutils/fcompiler/gnu.py

    python setup.py build
    sudo python setup.py install
{'fraction_non_alphanumeric': 0.13035532994923857, 'fraction_numerical': 0.00649746192893401, 'mean_word_length': 2.9408, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2669735', 'n_tokens_mistral': 1712, 'n_tokens_neox': 1621, 'n_words': 549}