<commit_before># Offline part

We have two Docker containers running in the [pod](k8s-offlinereporting.yaml):

- The [Web UI](https://hub.docker.com/r/mhausenblas/tsdemo-offline-reporting-ui/) 
- The [S3 fetcher](https://hub.docker.com/r/mhausenblas/tsdemo-s3-fetcher/)

The offline reporting Web UI and the S3 fetcher have a shared data volume at `/tmp/tsdemo`.

## Install offline reporting

The offline part is deployed via [Kubernetes](https://docs.mesosphere.com/services/kubernetes/), so let's install that:

    $ dcos config prepend package.sources https://github.com/mesosphere/multiverse/archive/version-1.x.zip
    $ dcos package update --validate
    $ dcos package install kubernetes

Now we need to configure Kubernetes:

    $ export KUBERNETES_MASTER=http://$MESOS_MASTER_IP/service/kubernetes/api

Next, [install](https://docs.mesosphere.com/services/kubernetes/#a-namefivealaunch-a-kubernetes-pod-and-service-by-using-kubectl) `kubectl`
and check if everything is fine:

    $  kubectl cluster-info
    Kubernetes master is running at http://172.16.17.32
    KubeDNS is running at http://172.16.17.32/api/v1/proxy/namespaces/kube-system/services/kube-dns
    KubeUI is running at http://172.16.17.32/api/v1/proxy/namespaces/kube-system/services/kube-ui
    
In order to work properly you'll need to supply the S3 fetcher with your AWS credentials, so create a file called `aws-secret.yaml` with this content:

    apiVersion: v1
    kind: Secret
    metadata:
      name: awssecret
    type: Opaque
    data:
      AWS-ACCESS-KEY-ID: $YOUR_AWS_KEY
      AWS-SECRET-ACCESS-KEY: $YOUR_AWS_SECRET

Make sure you replace `$YOUR_AWS_KEY` and `$YOUR_AWS_SECRET` with the `base64` encoded values you glean from the AWS IAM, see also this [docs](http://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). For example: `echo -n 123456789 | base64`.


Finally we can launch our offline reporting [replication controller](http://kubernetes.io/v1.0/docs/user-guide/walkthrough/k8s201.html#replication-controllers):

    $ kubectl create -f aws-secret.yaml
    $ kubectl create -f k8s-offlinereporting-rc.yaml
    $ kubectl get rc
    CONTROLLER            CONTAINER(S)   IMAGE(S)                                  SELECTOR                REPLICAS
    offlinereporting-rc   s3-fetcher     mhausenblas/tsdemo-s3-fetcher             name=offlinereporting   1
                          webui          mhausenblas/tsdemo-offline-reporting-ui
    $ kubectl create -f k8s-offlinereporting-service.yaml
    $ kubectl get endpoints
    NAME                       ENDPOINTS
    k8sm-scheduler             10.0.3.201:25504
    kubernetes                 10.0.3.201:25502
    offlinereporting-service   <none>

![K8S deployment](../img/k8s-deployment.png)

Once you're done, you can remove the RC with `kubectl delete rc offlinereporting-rc`.

In order to access the Web UI all you have to do is visit the following URL (with `$DCOS_DASHBOARD_FQHN` being your DCOS cluster dashboard URL):

    http://$DCOS_DASHBOARD_FQHN/service/kubernetes/api/v1/proxy/namespaces/default/services/offlinereporting-service/

![Offline reporting Web UI](../img/offline-reporting.png)<commit_msg>minor fix<commit_after># Offline part

We have two Docker containers running in the [pod](k8s-offlinereporting.yaml):

- The [Web UI](https://hub.docker.com/r/mhausenblas/tsdemo-offline-reporting-ui/) 
- The [S3 fetcher](https://hub.docker.com/r/mhausenblas/tsdemo-s3-fetcher/)

The offline reporting Web UI and the S3 fetcher have a shared data volume at `/tmp/tsdemo`.

## Install offline reporting

The offline part is deployed via [Kubernetes](https://docs.mesosphere.com/services/kubernetes/), so let's install that:

    $ dcos config prepend package.sources https://github.com/mesosphere/multiverse/archive/version-1.x.zip
    $ dcos package update --validate
    $ dcos package install kubernetes

Now we need to configure Kubernetes:

    $ export KUBERNETES_MASTER=http://$MESOS_MASTER_IP/service/kubernetes/api

Next, [install](https://docs.mesosphere.com/services/kubernetes/#a-namefivealaunch-a-kubernetes-pod-and-service-by-using-kubectl) `kubectl`
and check if everything is fine:

    $  kubectl cluster-info
    Kubernetes master is running at http://172.16.17.32
    KubeDNS is running at http://172.16.17.32/api/v1/proxy/namespaces/kube-system/services/kube-dns
    KubeUI is running at http://172.16.17.32/api/v1/proxy/namespaces/kube-system/services/kube-ui
    
In order to work properly you'll need to supply the S3 fetcher with your AWS credentials, so create a file called `aws-secret.yaml` with this content:

    apiVersion: v1
    kind: Secret
    metadata:
      name: awssecret
    type: Opaque
    data:
      AWS-ACCESS-KEY-ID: $YOUR_AWS_KEY
      AWS-SECRET-ACCESS-KEY: $YOUR_AWS_SECRET

Make sure you replace `$YOUR_AWS_KEY` and `$YOUR_AWS_SECRET` with the `base64` encoded values you glean from the AWS IAM, see also this [docs](http://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). For example: `echo -n 123456789 | base64`.


Finally we can launch our offline reporting [replication controller](http://kubernetes.io/v1.0/docs/user-guide/walkthrough/k8s201.html#replication-controllers):

    $ kubectl create -f aws-secret.yaml
    $ kubectl create -f k8s-offlinereporting-rc.yaml
    $ kubectl get rc
    CONTROLLER            CONTAINER(S)   IMAGE(S)                                  SELECTOR                REPLICAS
    offlinereporting-rc   s3-fetcher     mhausenblas/tsdemo-s3-fetcher             name=offlinereporting   1
                          webui          mhausenblas/tsdemo-offline-reporting-ui
    $ kubectl create -f k8s-offlinereporting-service.yaml
    $ kubectl get endpoints
    NAME                       ENDPOINTS
    k8sm-scheduler             10.0.3.201:25504
    kubernetes                 10.0.3.201:25502
    offlinereporting-service   10.0.3.200:1028

![K8S deployment](../img/k8s-deployment.png)

Once you're done, you can remove the RC with `kubectl delete rc offlinereporting-rc`.

In order to access the Web UI all you have to do is visit the following URL (with `$DCOS_DASHBOARD_FQHN` being your DCOS cluster dashboard URL):

    http://$DCOS_DASHBOARD_FQHN/service/kubernetes/api/v1/proxy/namespaces/default/services/offlinereporting-service/

![Offline reporting Web UI](../img/offline-reporting.png)
{'fraction_non_alphanumeric': 0.10195465207193119, 'fraction_numerical': 0.029241594996090695, 'mean_word_length': 3.1078998073217727, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 11, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5240983', 'n_tokens_mistral': 2299, 'n_tokens_neox': 2093, 'n_words': 510}