<commit_before>.. index:: installation

.. _installation-chapter:

Installation
============

Socorro VM (built with Vagrant + Puppet)
------------

You can build a standalone Socorro development VM -
see :ref:`setupdevenv-chapter` for more info. 

The config files and puppet manifests in ./puppet/ are a useful reference
when setting up Socorro for the first time.

Automated Install using Puppet
------------

It is possible to use puppet to script an install onto an existing environment.
This has been tested in EC2 but should work on any regular Ubuntu Lucid install.

See puppet/bootstrap.sh for an example.

Manual Install
------------

Requirements
````````````

.. sidebar:: Breakpad client and symbols

   Socorro aggregates and reports on Breakpad crashes.
   Read more about `getting started with Breakpad <http://code.google.com/p/google-breakpad/wiki/GettingStartedWithBreakpad>`_.

   You will need to `produce symbols for your application <http://code.google.com/p/google-breakpad/wiki/LinuxStarterGuide#Producing_symbols_for_your_application>`_ and make these files available to Socorro.

* Linux (tested on Ubuntu Lucid and RHEL/CentOS 6)

* HBase (Cloudera CDH3)

* PostgreSQL 9.0

* Python 2.6

Ubuntu
````````````
1) Add PostgreSQL 9.0 PPA from https://launchpad.net/~pitti/+archive/postgresql
2) Add Cloudera apt source from https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation#CDH3Installation-InstallingCDH3onUbuntuSystems
3) Install dependencies using apt-get

As *root*:
::
  apt-get install supervisor rsyslog libcurl4-openssl-dev build-essential sun-java6-jdk ant python-software-properties subversion libpq-dev python-virtualenv python-dev libcrypt-ssleay-perl phpunit php5-tidy python-psycopg2 python-simplejson apache2 libapache2-mod-wsgi memcached php5-pgsql php5-curl php5-dev php-pear php5-common php5-cli php5-memcache php5 php5-gd php5-mysql php5-ldap hadoop-hbase hadoop-hbase-master hadoop-hbase-thrift curl liblzo2-dev postgresql-9.0 postgresql-plperl-9.0 postgresql-contrib

RHEL/Centos
````````````
Use "text install"
Choose "minimal" as install option.

1) Add Cloudera yum repo from https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation#CDH3Installation-InstallingCDH3onRedHatSystems
2) Add PostgreSQL 9.0 yum repo from http://www.postgresql.org/download/linux#yum
3) Install Sun Java JDK version JDK 6u16 - Download appropriate package from http://www.oracle.com/technetwork/java/javase/downloads/index.html
4) Install dependencies using YUM:

As *root*:
::
  yum install python-psycopg2 simplejson httpd mod_ssl mod_wsgi postgresql-server postgresql-plperl perl-pgsql_perl5 postgresql-contrib subversion make rsync php-pecl-memcache memcached php-pgsql subversion gcc-c++ curl-devel ant python-virtualenv php-phpunit-PHPUnit hadoop-0.20 hadoop-hbase daemonize

5) Disable SELinux

As *root*:
  Edit /etc/sysconfig/selinux and set "SELINUX=disabled"

6) Reboot

As *root*:
::
  shutdown -r now

Download and install Socorro
````````````
Determine latest release tag from https://wiki.mozilla.org/Socorro:Releases#Previous_Releases

Clone from github, as the *socorro* user:
::
  git clone https://github.com/mozilla/socorro
  git checkout LATEST_RELEASE_TAG_GOES_HERE
  cd socorro
  cp scripts/config/commonconfig.py.dist scripts/config/commonconfig.py

Edit scripts/config/commonconfig.py

From inside the Socorro checkout, as the *socorro* user, change:
::
  databaseName.default = 'breakpad'
  databaseUserName.default = 'breakpad_rw'
  databasePassword.default = '<PASSWORD>'

If you change the password, make sure to change it in sql/roles.sql as well.

Run unit/functional tests, and generate report
````````````
From inside the Socorro checkout, as the *socorro* user:
::
  make coverage

Set up directories and permissions
````````````
As *root*:
::
  mkdir /etc/socorro
  mkdir /var/log/socorro
  mkdir -p /data/socorro
  useradd socorro
  chown socorro:socorro /var/log/socorro
  mkdir /home/socorro/primaryCrashStore /home/socorro/fallback
  chown apache /home/socorro/primaryCrashStore /home/socorro/fallback
  chmod 2775 /home/socorro/primaryCrashStore /home/socorro/fallback

Note - use www-data instead of apache for debian/ubuntu

Compile minidump_stackwalk

From inside the Socorro checkout, as the *socorro* user:
::
  make minidump_stackwalk<commit_msg>(docs only) - note about needing install-submodules, and make test is fine to run standlone now instead of coverage<commit_after>.. index:: installation

.. _installation-chapter:

Installation
============

Socorro VM (built with Vagrant + Puppet)
------------

You can build a standalone Socorro development VM -
see :ref:`setupdevenv-chapter` for more info. 

The config files and puppet manifests in ./puppet/ are a useful reference
when setting up Socorro for the first time.

Automated Install using Puppet
------------

It is possible to use puppet to script an install onto an existing environment.
This has been tested in EC2 but should work on any regular Ubuntu Lucid install.

See puppet/bootstrap.sh for an example.

Manual Install
------------

Requirements
````````````

.. sidebar:: Breakpad client and symbols

   Socorro aggregates and reports on Breakpad crashes.
   Read more about `getting started with Breakpad <http://code.google.com/p/google-breakpad/wiki/GettingStartedWithBreakpad>`_.

   You will need to `produce symbols for your application <http://code.google.com/p/google-breakpad/wiki/LinuxStarterGuide#Producing_symbols_for_your_application>`_ and make these files available to Socorro.

* Linux (tested on Ubuntu Lucid and RHEL/CentOS 6)

* HBase (Cloudera CDH3)

* PostgreSQL 9.0

* Python 2.6

Ubuntu
````````````
1) Add PostgreSQL 9.0 PPA from https://launchpad.net/~pitti/+archive/postgresql
2) Add Cloudera apt source from https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation#CDH3Installation-InstallingCDH3onUbuntuSystems
3) Install dependencies using apt-get

As *root*:
::
  apt-get install supervisor rsyslog libcurl4-openssl-dev build-essential sun-java6-jdk ant python-software-properties subversion libpq-dev python-virtualenv python-dev libcrypt-ssleay-perl phpunit php5-tidy python-psycopg2 python-simplejson apache2 libapache2-mod-wsgi memcached php5-pgsql php5-curl php5-dev php-pear php5-common php5-cli php5-memcache php5 php5-gd php5-mysql php5-ldap hadoop-hbase hadoop-hbase-master hadoop-hbase-thrift curl liblzo2-dev postgresql-9.0 postgresql-plperl-9.0 postgresql-contrib

RHEL/Centos
````````````
Use "text install"
Choose "minimal" as install option.

1) Add Cloudera yum repo from https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation#CDH3Installation-InstallingCDH3onRedHatSystems
2) Add PostgreSQL 9.0 yum repo from http://www.postgresql.org/download/linux#yum
3) Install Sun Java JDK version JDK 6u16 - Download appropriate package from http://www.oracle.com/technetwork/java/javase/downloads/index.html
4) Install dependencies using YUM:

As *root*:
::
  yum install python-psycopg2 simplejson httpd mod_ssl mod_wsgi postgresql-server postgresql-plperl perl-pgsql_perl5 postgresql-contrib subversion make rsync php-pecl-memcache memcached php-pgsql subversion gcc-c++ curl-devel ant python-virtualenv php-phpunit-PHPUnit hadoop-0.20 hadoop-hbase daemonize

5) Disable SELinux

As *root*:
  Edit /etc/sysconfig/selinux and set "SELINUX=disabled"

6) Reboot

As *root*:
::
  shutdown -r now

Download and install Socorro
````````````
Determine latest release tag from https://wiki.mozilla.org/Socorro:Releases#Previous_Releases

Clone from github, as the *socorro* user:
::
  git clone https://github.com/mozilla/socorro
  git checkout LATEST_RELEASE_TAG_GOES_HERE
  cd socorro
  cp scripts/config/commonconfig.py.dist scripts/config/commonconfig.py

Edit scripts/config/commonconfig.py

From inside the Socorro checkout, as the *socorro* user, change:
::
  databaseName.default = 'breakpad'
  databaseUserName.default = 'breakpad_rw'
  databasePassword.default = '<PASSWORD>'

If you change the password, make sure to change it in sql/roles.sql as well.

Run unit/functional tests, and generate report
````````````
From inside the Socorro checkout, as the *socorro* user:
::
  # only need install-submodules for pre-9.0 versions of Socorro
  make install-submodules
  make test

Set up directories and permissions
````````````
As *root*:
::
  mkdir /etc/socorro
  mkdir /var/log/socorro
  mkdir -p /data/socorro
  useradd socorro
  chown socorro:socorro /var/log/socorro
  mkdir /home/socorro/primaryCrashStore /home/socorro/fallback
  chown apache /home/socorro/primaryCrashStore /home/socorro/fallback
  chmod 2775 /home/socorro/primaryCrashStore /home/socorro/fallback

Note - use www-data instead of apache for debian/ubuntu

Compile minidump_stackwalk

From inside the Socorro checkout, as the *socorro* user:
::
  make minidump_stackwalk
{'fraction_non_alphanumeric': 0.10451306413301663, 'fraction_numerical': 0.013573125212080081, 'mean_word_length': 4.56801007556675, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 10, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3998381', 'n_tokens_mistral': 3113, 'n_tokens_neox': 2780, 'n_words': 854}