<commit_before>===================================
Setting up a new Push Test Instance
===================================

These instructions will help you setup a new Pushgo test instance on AWS. A
single AWS t2.micro can run all the following without too much whimpering.

AWS Setup
=========

1) Choose CoreOS Stable, latest version as AMI
2) Create instance XXX or better
3) Configure Security Group ports to allow from any IP
    - 8090/tcp  Push Websocket
    - 8081/tcp  Push Endpoint
    - 8086/tcp  InfluxDB REST Endpoint
    - 8083/tcp  InfluxDB Admin UI
    - 8000/tcp  Grafana
    - 8080/tcp  Kibana
    - 9200/tcp  ElasticSearch
4) Start with chosen SSH keypair

Machine Setup
=============

SSH into the instance, where you will be running the remaining commands.

Setup the environment var ``PUBLIC_IP`` to refer to the AWS instances public IP:
PUBLIC_IP=AWS_PUBLIC_IP_HERE

Copy/Paste the following, these instructions will pull the appropriate containers and run them:

..


    docker pull minimum2scp/es-kibana:latest
    docker pull kitcambridge/heka:dev
    docker pull kitcambridge/cadvisor:influxdb
    docker pull bbangert/pushgo:1.4rc5
    docker pull tutum/influxdb
    docker pull tutum/grafana

    INFLUX_CID=$(docker run -d -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 -e PRE_CREATE_DB="pushgo" tutum/influxdb)
    INFLUX_IP=$(docker inspect $INFLUX_CID | grep IPAddress | cut -d '"' -f 4)

    docker run -d -p 8000:80 -e INFLUXDB_HOST=$PUBLIC_IP -e INFLUXDB_PORT=8086 -e INFLUXDB_NAME=pushgo -e INFLUXDB_USER=root \
        -e INFLUXDB_PASS=<PASSWORD> -e INFLUXDB_IS_GRAFANADB=true -e HTTP_USER=admin -e HTTP_PASS=<PASSWORD> tutum/grafana


    ELASTIC_CID=$(docker run -d -p 8080:80 -p 9200:9200 minimum2scp/es-kibana)
    ELASTIC_IP=$(docker inspect $ELASTIC_CID | grep IPAddress | cut -d '"' -f 4)

    docker run -d --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro \
        kitcambridge/cadvisor:influxdb -storage_driver=influxdb -storage_driver_host=$INFLUX_IP:8086 -storage_driver_db=pushgo -storage_driver_buffer_duration=5.000000000s

    mkdir -p heka<commit_msg>Try another code-block.<commit_after>===================================
Setting up a new Push Test Instance
===================================

These instructions will help you setup a new Pushgo test instance on AWS. A
single AWS t2.micro can run all the following without too much whimpering.

AWS Setup
=========

1) Choose CoreOS Stable, latest version as AMI
2) Create instance XXX or better
3) Configure Security Group ports to allow from any IP
    - 8090/tcp  Push Websocket
    - 8081/tcp  Push Endpoint
    - 8086/tcp  InfluxDB REST Endpoint
    - 8083/tcp  InfluxDB Admin UI
    - 8000/tcp  Grafana
    - 8080/tcp  Kibana
    - 9200/tcp  ElasticSearch
4) Start with chosen SSH keypair

Machine Setup
=============

SSH into the instance, where you will be running the remaining commands.

Setup the environment var ``PUBLIC_IP`` to refer to the AWS instances public IP:
PUBLIC_IP=AWS_PUBLIC_IP_HERE

Copy/Paste the following, these instructions will pull the appropriate containers and run them:

.. code-block: text


    docker pull minimum2scp/es-kibana:latest
    docker pull kitcambridge/heka:dev
    docker pull kitcambridge/cadvisor:influxdb
    docker pull bbangert/pushgo:1.4rc5
    docker pull tutum/influxdb
    docker pull tutum/grafana

    INFLUX_CID=$(docker run -d -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 -e PRE_CREATE_DB="pushgo" tutum/influxdb)
    INFLUX_IP=$(docker inspect $INFLUX_CID | grep IPAddress | cut -d '"' -f 4)

    docker run -d -p 8000:80 -e INFLUXDB_HOST=$PUBLIC_IP -e INFLUXDB_PORT=8086 -e INFLUXDB_NAME=pushgo -e INFLUXDB_USER=root \
        -e INFLUXDB_PASS=<PASSWORD> -e INFLUXDB_IS_GRAFANADB=true -e HTTP_USER=admin -e HTTP_PASS=<PASSWORD> tutum/grafana


    ELASTIC_CID=$(docker run -d -p 8080:80 -p 9200:9200 minimum2scp/es-kibana)
    ELASTIC_IP=$(docker inspect $ELASTIC_CID | grep IPAddress | cut -d '"' -f 4)

    docker run -d --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro \
        kitcambridge/cadvisor:influxdb -storage_driver=influxdb -storage_driver_host=$INFLUX_IP:8086 -storage_driver_db=pushgo -storage_driver_buffer_duration=5.000000000s

    mkdir -p heka
{'fraction_non_alphanumeric': 0.12463042983852626, 'fraction_numerical': 0.04639526950193314, 'mean_word_length': 3.843612334801762, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2719805', 'n_tokens_mistral': 1720, 'n_tokens_neox': 1479, 'n_words': 447}