<commit_before>These instructions assume Debian Squeeze or Ubuntu 11.04, or later
(probably, though we won't necessarily have tested in later versions
yet!)
[Install instructions for OS X](https://github.com/sebbacon/alaveteli/wiki/OS-X-Quickstart)
are under development.

It is possible to install on Ubuntus as old as 10.04, but you must use
[Xapian backports](https://launchpad.net/~xapian-backports/+archive/xapian-1.2)
(see [issue #158](https://github.com/sebbacon/alaveteli/issues/159)
for discussion).

Commands are intended to be run via the terminal or over ssh.

As an aid to evaluation, there is an
[Amazon AMI](https://github.com/sebbacon/alaveteli/wiki/Alaveteli-ec2-amix)
with all these steps configured.  It is *not* production-ready.

# Package Installation

To start with, you may need to install git, e.g. with `sudo apt-get
install git-core`

Next, get hold of the Alaveteli source code from github: 

    git clone https://github.com/sebbacon/alaveteli.git
    cd alaveteli

Install the packages that are listed in config/packages using apt-get e.g.:

    sudo apt-get install `cut -d " " -f 1 config/packages | grep -v "^#"`

Some of the files also have a version number listed in config/packages - check
that you have appropriate versions installed. Some also list "|" and offer
a choice of packages.

# Ruby dependency installation

Install rubygems 1.6.1 (we're not using the Debian package because we
want an old version; see "Troubleshooting" below for an explanation
why):

    cd /tmp/
    wget http://rubyforge.org/frs/download.php/74445/rubygems-1.6.2.tgz
    tar zxvf /tmp/rubygems-1.6.2.tgz -C /tmp/
    sudo ruby /tmp/rubygems-1.6.2/setup.rb
 
Now, to install Alaveteli's Ruby dependencies, we need to install bundler:

    sudo gem install bundler
    bundle install --deployment

On Debian, at least, the binaries installed by bundler are not put in
the system `PATH`; therefore, in order to run `rake`, you will need to
do something like:

    ln -s /usr/lib/ruby/gems/1.8/bin/rake /usr/local/bin/

You will also want to install mySociety's common ruby libraries and the Rails
code. Run:

    git submodule update --init

to fetch the contents of the submodules.

If you would like users to be able to download pretty PDFs as part of 
the downloadable zipfile of their request history, you should also install
[wkhtmltopdf](http://code.google.com/p/wkhtmltopdf/downloads/list).  
We recommend downloading the latest, statically compiled version from
the project website, as this allows running headless (i.e. without a
graphical interface running) on Linux.  If you do install
`wkhtmltopdf`, you need to edit a setting in the config file to point
to it (see below).  If you don't install it, everything will still work, but 
users will get ugly, plain text versions of their requests when they 
download them.

Version 1.44 of `pdftk` contains a bug which makes it to loop forever
in certain edge conditions.  Until it's incorporated into an official
release, you can either hope you don't encounter the bug (it ties up a
rails process until you kill it) you'll need to patch it yourself or
use the Debian package compiled by mySociety (see link in
[issue 305](https://github.com/sebbacon/alaveteli/issues/305))

# Configure Database 

There has been a little work done in trying to make the code work with
other databases (e.g. SQLite), but the preferred database is PostgreSQL.

If you don't have it installed:

    apt-get install postgresql postgresql-client

Now we need to set up the database config file to contain the name,
username and password of your postgres database.

* copy `database.yml-example` to `database.yml` in `alaveteli/config`
* edit it to point to your local postgresql database in the development
  and test sections and create the databases:

Make sure that the user specified in database.yml exists, and has full
permissions on these databases. As they need the ability to turn off
constraints whilst running the tests they also need to be a superuser.
(See http://dev.rubyonrails.org/ticket/9981)
  
You can create a `foi` user from the command line, thus:

    # su - postgres
    $ createuser -s -P foi
    
And you can create a database thus:

    $ createdb -T template0 -E SQL_ASCII -O foi foi_production
    $ createdb -T template0 -E SQL_ASCII -O foi foi_test
    $ createdb -T template0 -E SQL_ASCII -O foi foi_development

We create using the ``SQL_ASCII`` encoding, because in postgres this
is means "no encoding"; and because we handle and store all kinds of
data that may not be valid UTF (for example, data originating from
various broken email clients that's not 8-bit clean), it's safer to be
able to store *anything*, than reject data at runtime.

# Configure email

You will need to set up an email server (MTA) to send and receive
emails.  Full configuration for an MTA is beyond the scope of this
document. However, just to get the tests to pass, you will at a
minimum need to allow sending emails via a `sendmail` command (a
requirement met, for example, with `sudo apt-get install exim4`).

To receive email in a production setup, you will also need to
configure your MTA to pipe incoming emails to the Alaveteli script
`script/mailin`.  An example configuration is described in
`INSTALL-exim4.md`.

# Set up configs

opy `config/general.yml-example` to `config/general.yml` and edit to
your taste.

Note that the default settings for frontpage examples are designed to
work with the dummy data shipped with Alaveteli; once you have real
data, you should edit these.

The default theme is the "Alaveteli" theme.  When you run
`rails-post-deploy` (see below), that theme gets installed
automatically.

You'll also want to copy `config/memcached.yml-example` to
`config/memcached.yml`. The application is configured, via the
Interlock Rails plugin, to cache content using memcached.  You
probably don't want this in your development profile; the example
`memcached.yml` file disables this behaviour.

# Deployment

In the 'alaveteli' directory, run:

    ./script/rails-post-deploy 

(This will need execute privs so `chmod 755` if necessary.) This sets
up directory structures, creates logs, installs/updates themes, runs
database migrations, etc.  You should run it after each new software<commit_msg>Some more INSTALL corrections and clarifications.<commit_after>These instructions assume Debian Squeeze or Ubuntu 11.04, or later
(probably, though we won't necessarily have tested in later versions
yet!)
[Install instructions for OS X](https://github.com/sebbacon/alaveteli/wiki/OS-X-Quickstart)
are under development.

It is possible to install on Ubuntus as old as 10.04, but you must use
[Xapian backports](https://launchpad.net/~xapian-backports/+archive/xapian-1.2)
(see [issue #158](https://github.com/sebbacon/alaveteli/issues/159)
for discussion).

Commands are intended to be run via the terminal or over ssh.

As an aid to evaluation, there is an
[Amazon AMI](https://github.com/sebbacon/alaveteli/wiki/Alaveteli-ec2-amix)
with all these steps configured.  It is *not* production-ready.

# Package Installation

To start with, you may need to install git, e.g. with `sudo apt-get
install git-core`

Next, get hold of the Alaveteli source code from github: 

    git clone https://github.com/sebbacon/alaveteli.git
    cd alaveteli

If you are running Debian, you can use some of the specially compiled
mysociety packages by adding the following to /etc/apt/sources.list:

    deb http://debian.mysociety.org squeeze main non-free contrib

Install the packages that are listed in config/packages using apt-get e.g.:

    sudo apt-get install `cut -d " " -f 1 config/packages | grep -v "^#"`

Some of the files also have a version number listed in config/packages - check
that you have appropriate versions installed. Some also list "|" and offer
a choice of packages.

# Ruby dependency installation

Install rubygems 1.6.1 (we're not using the Debian package because we
want an old version; see "Troubleshooting" below for an explanation
why):

    wget http://rubyforge.org/frs/download.php/74445/rubygems-1.6.2.tgz -O /tmp/rubygems-1.6.2.tgz
    tar zxvf /tmp/rubygems-1.6.2.tgz -C /tmp/
    sudo ruby /tmp/rubygems-1.6.2/setup.rb
 
Now, to install Alaveteli's Ruby dependencies, we need to install bundler:

    sudo gem1.8 install bundler
    bundle install --deployment
    
The part of the `bundle install` that compiles `xapian-full` takes a
*long* time!

You will also want to install mySociety's common ruby libraries and the Rails
code. Run:

    git submodule update --init

to fetch the contents of the submodules.

## Packages customised by mySociety

Debian users should add the mySociety debian archive to their
`/etc/apt/sources.list` as described above.  Doing this and following
the above instructions should install a couple of custom
dependencies. Users of other platforms will have to install these
dependencies manually, as follows:

1. If you would like users to be able to download pretty PDFs as part of
the downloadable zipfile of their request history, you should install
[wkhtmltopdf](http://code.google.com/p/wkhtmltopdf/downloads/list).
We recommend downloading the latest, statically compiled version from
the project website, as this allows running headless (i.e. without a
graphical interface running) on Linux.  If you do install
`wkhtmltopdf`, you need to edit a setting in the config file to point
to it (see below).  If you don't install it, everything will still
work, but users will get ugly, plain text versions of their requests
when they download them.

2. Version 1.44 of `pdftk` contains a bug which makes it to loop forever
in certain edge conditions.  Until it's incorporated into an official
release, you can either hope you don't encounter the bug (it ties up a
rails process until you kill it) you'll need to patch it yourself or
use the Debian package compiled by mySociety (see link in
[issue 305](https://github.com/sebbacon/alaveteli/issues/305))


# Configure Database 

There has been a little work done in trying to make the code work with
other databases (e.g. SQLite), but the preferred database is PostgreSQL.

If you don't have it installed:

    apt-get install postgresql postgresql-client

Now we need to set up the database config file to contain the name,
username and password of your postgres database.

* copy `database.yml-example` to `database.yml` in `alaveteli/config`
* edit it to point to your local postgresql database in the development
  and test sections and create the databases:

Make sure that the user specified in database.yml exists, and has full
permissions on these databases. As they need the ability to turn off
constraints whilst running the tests they also need to be a superuser.
(See http://dev.rubyonrails.org/ticket/9981)
  
You can create a `foi` user from the command line, thus:

    # su - postgres
    $ createuser -s -P foi
    
And you can create a database thus:

    $ createdb -T template0 -E SQL_ASCII -O foi foi_production
    $ createdb -T template0 -E SQL_ASCII -O foi foi_test
    $ createdb -T template0 -E SQL_ASCII -O foi foi_development

We create using the ``SQL_ASCII`` encoding, because in postgres this
is means "no encoding"; and because we handle and store all kinds of
data that may not be valid UTF (for example, data originating from
various broken email clients that's not 8-bit clean), it's safer to be
able to store *anything*, than reject data at runtime.

# Configure email

You will need to set up an email server (MTA) to send and receive
emails.  Full configuration for an MTA is beyond the scope of this
document. However, just to get the tests to pass, you will at a
minimum need to allow sending emails via a `sendmail` command (a
requirement met, for example, with `sudo apt-get install exim4`).

To receive email in a production setup, you will also need to
configure your MTA to pipe incoming emails to the Alaveteli script
`script/mailin`.  An example configuration is described in
`INSTALL-exim4.md`.

# Set up configs

opy `config/general.yml-example` to `config/general.yml` and edit to
your taste.

Note that the default settings for frontpage examples are designed to
work with the dummy data shipped with Alaveteli; once you have real
data, you should edit these.

The default theme is the "Alaveteli" theme.  When you run
`rails-post-deploy` (see below), that theme gets installed
automatically.

You'll also want to copy `config/memcached.yml-example` to
`config/memcached.yml`. The application is configured, via the
Interlock Rails plugin, to cache content using memcached.  You
probably don't want this in your development profile; the example
`memcached.yml` file disables this behaviour.

# Deployment

On Debian, at least, the binaries installed by bundler are not put in
the system `PATH`; therefore, in order to run `rake` (needed for
deployments), you will need to do something like:

    ln -s /usr/lib/ruby/gems/1.8/bin/rake /usr/local/bin/

Now, in the 'alaveteli' directory, run:

    ./script/rails-post-deploy 

(This will need execute privs so `chmod 755` if necessary.) This sets
up directory structures, creates logs, installs/updates themes, runs
database migrations, etc.  You should run it after each new software
{'fraction_non_alphanumeric': 0.06808254255158909, 'fraction_numerical': 0.00941406838379274, 'mean_word_length': 4.018518518518518, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6149343', 'n_tokens_mistral': 4005, 'n_tokens_neox': 3744, 'n_words': 1868}