<commit_before># zeit.recommend

This guide describes how to setup the Zeit Recommend service.    
(Most commands must be run as root.)

Currently two machines are needed:

* Varnish caching server
* RabbitMQ queuing server

-------------------------------------------------------------------------------

## Varnish

* Install `openbsd-inetd` or equivalent

```shell
apt-get install openbsd-inetd
```

* Configure inetd for varnishncsa

```shell
cat varnish/etc/inetd.conf >> /etc/inetd.conf
```

* Reload the inetd config

```shell
sudo /etc/init.d/openbsd-inetd restart
```

* Allow TCP traffic on port 3000

## Rabbit

* Add a new user for logstash

```shell
useradd -m logstash
```

* Copy the logstash config file

```shell
mkdir /etc/logstash && cp logstash/etc/logstash/agent.conf /etc/logstash/
```

* Paste the varnish host name into this line in /etc/logstash/agent.conf

```shell
host => "{INSERT HOST URL HERE}"
```

* Download logstash jar file

```shell
curl https://download.elasticsearch.org/logstash/logstash/logstash-1.2.2-flatjar.jar \
/home/logstash/logstash.jar
```

* Add the following line to your /etc/apt/sources.list

```shell
deb http://www.rabbitmq.com/debian/ testing main
```

* Trust the rabbitmq public key

```shell
wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
apt-key add rabbitmq-signing-key-public.asc
```

* Update package index

```shell
apt-get update
```

* Install required packages

```shell
apt-get install openjdk-6-jre python-dev python-setuptools git rabbitmq-server upstart
```

* Ensure rabbit is running

```shell
rabbitmqctl status
```
<commit_msg>Remove git from dependency list<commit_after># zeit.recommend

This guide describes how to setup the Zeit Recommend service.    
(Most commands must be run as root.)

Currently two machines are needed:

* Varnish caching server
* RabbitMQ queuing server

-------------------------------------------------------------------------------

## Varnish

* Install `openbsd-inetd` or equivalent

```shell
apt-get install openbsd-inetd
```

* Configure inetd for varnishncsa

```shell
cat varnish/etc/inetd.conf >> /etc/inetd.conf
```

* Reload the inetd config

```shell
sudo /etc/init.d/openbsd-inetd restart
```

* Allow TCP traffic on port 3000

## Rabbit

* Add a new user for logstash

```shell
useradd -m logstash
```

* Copy the logstash config file

```shell
mkdir /etc/logstash && cp logstash/etc/logstash/agent.conf /etc/logstash/
```

* Paste the varnish host name into this line in /etc/logstash/agent.conf

```shell
host => "{INSERT HOST URL HERE}"
```

* Download logstash jar file

```shell
curl https://download.elasticsearch.org/logstash/logstash/logstash-1.2.2-flatjar.jar \
/home/logstash/logstash.jar
```

* Add the following line to your /etc/apt/sources.list

```shell
deb http://www.rabbitmq.com/debian/ testing main
```

* Trust the rabbitmq public key

```shell
wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
apt-key add rabbitmq-signing-key-public.asc
```

* Update package index

```shell
apt-get update
```

* Install required packages

```shell
apt-get install openjdk-6-jre python-dev python-setuptools rabbitmq-server upstart
```

* Ensure rabbit is running

```shell
rabbitmqctl status
```

{'fraction_non_alphanumeric': 0.16707768187422933, 'fraction_numerical': 0.004932182490752158, 'mean_word_length': 3.2252604166666665, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 9, 'https://': 2, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2665883', 'n_tokens_mistral': 1160, 'n_tokens_neox': 1057, 'n_words': 313}