<commit_before># Overview

InfluxDB is an open source database for storing time series data. The source of time series data may come from logging and monitoring systems and IoT devices.

This is a single-instance version of InfluxDB. Multi-instance version of InfluxDB requires commercial license.

If you are interested in enterprise version of InfluxDB or you would like to learn more about InfluxDB in general, please, visit [InfluxDB website](https://www.influxdata.com/).

## About Google Click to Deploy

Popular open stacks on Kubernetes packaged by Google.

# Installation

## Quick install with Google Cloud Marketplace

Get up and running with a few clicks! Install this InfluxDB app to a
Google Kubernetes Engine cluster using Google Cloud Marketplace. Follow the
[on-screen instructions](https://console.cloud.google.com/launcher/details/google/influxdb).

## Command line instructions

### Prerequisites

#### Set up command-line tools

You'll need the following tools in your development environment:
- [gcloud](https://cloud.google.com/sdk/gcloud/)
- [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/)
- [docker](https://docs.docker.com/install/)

#### Create a Google Kubernetes Engine cluster

Create a new cluster from the command-line.

```shell
export CLUSTER=marketplace-cluster
export ZONE=us-west1-a

gcloud container clusters create "$CLUSTER" --zone "$ZONE"
```

Configure `kubectl` to talk to the new cluster.

```shell
gcloud container clusters get-credentials "$CLUSTER" --zone "$ZONE"
```

#### Clone this repo

Clone this repo and the associated tools repo.

```shell
gcloud source repos clone google-click-to-deploy --project=k8s-marketplace-eap
gcloud source repos clone google-marketplace-k8s-app-tools --project=k8s-marketplace-eap
```

#### Install the Application resource definition

Do a one-time setup for your cluster to understand Application resources.

<!--
To do that, navigate to `k8s/vendor` subdirectory of the repository and run the following command:
-->

```shell
kubectl apply -f google-marketplace-k8s-app-tools/crd/*
```

The Application resource is defined by the
[Kubernetes SIG-apps](https://github.com/kubernetes/community/tree/master/sig-apps)
community. The source code can be found on
[github.com/kubernetes-sigs/application](https://github.com/kubernetes-sigs/application).

### Install the Application

Navigate to the `influxdb` directory.

```shell
cd google-click-to-deploy/k8s/influxdb
```

#### Configure the app with environment variables

Choose the instance name and namespace for the app.

```shell
export APP_INSTANCE_NAME=influxdb-1
export NAMESPACE=default
```

Configure the container images.

```shell
export IMAGE_INFLUXDB="gcr.io/k8s-marketplace-eap/google/influxdb:latest"
```

Configure InfluxDB administrator account:

```shell
export INFLUXDB_ADMIN_USER=influxdb-admin
```

Configure password for InfluxDB administrator account (the value has to be encoded in base64)

```shell
export INFLUXDB_ADMIN_PASSWORD=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 12 | head -n 1 | tr -d '\n' | base64)
```

The images above are referenced by
[tag](https://docs.docker.com/engine/reference/commandline/tag). It is strongly
recommended to pin each image to an immutable
[content digest](https://docs.docker.com/registry/spec/api/#content-digests).
This will ensure that the installed application will always use the same images,
until you are ready to upgrade.

```shell
repo=`echo $IMAGE_INFLUXDB | cut -d: -f1`;
digest=`docker pull $IMAGE_INFLUXDB | sed -n -e 's/Digest: //p'`;
export $i="$repo@$digest";
env | grep $i;
```

#### Expand the manifest template

Use `envsubst` to expand the template. It is recommended that you save the
expanded manifest file for future updates to the application.

```shell
awk 'BEGINFILE {print "---"}{print}' manifest/* \
  | envsubst '$APP_INSTANCE_NAME $NAMESPACE $IMAGE_INFLUXDB $INFLUXDB_ADMIN_USER $INFLUXDB_ADMIN_PASSWORD' \
  > "${APP_INSTANCE_NAME}_manifest.yaml"
```

#### Apply to Kubernetes

Use `kubectl` to apply the manifest to your Kubernetes cluster.

```shell
kubectl apply -f "${APP_INSTANCE_NAME}_manifest.yaml" --namespace "${NAMESPACE}"
```

#### View the app in the Google Cloud Console

Point your browser to:

```shell
echo "https://console.cloud.google.com/kubernetes/application/${ZONE}/${CLUSTER}/${NAMESPACE}/${APP_INSTANCE_NAME}"
```

# Basic Usage

TODO by rafalbiegacz@ after merging

Run the following command to discover IP address of InfluxDB instance using kubectl:

```shell
kubectl get svc -o wide -l app.kubernetes.io/name=$APP_INSTANCE_NAME --namespace "$NAMESPACE"
```

# Scaling

This is a single-instance version of InfluxDB. You cannot scale it.

If you are interested in multi-instance/enterprise version of InfluxDB, please, visit [InfluxDB website](https://www.influxdata.com/).

# Backup and Restore

TODO by rafalbiegacz@ after merging

# Upgrade

This is single-instance InfluxDB solution:
- Upgrade will case some downtime of your application
- Configuration and data of InfluxDB will be retained.

This procudure assumes that you have a new image for InfluxDB container published and being available to your Kubernetes cluster. The new image is available at <url-pointing-to-new-image>.

Start with modification of the image used for pod temaplate within InfluxDB StatefulSet:

```shell
kubectl set image statefulset "$APP_INSTANCE_NAME-influxdb" \
  influxdb=<url-pointing-to-new-image>
```

where `<url-pointing-to-new-image>` is the new image.

To check the status of Pods in the StatefulSet and the progress of deployment of new image run the following command:

```shell
kubectl get pods -l app.kubernetes.io/name=$APP_INSTANCE_NAME
```

To check the current image used by pods within `InfluxDB` K8s application, you can run the following command:

```shell
kubectl get pods -l app.kubernetes.io/name=$APP_INSTANCE_NAME -o=jsonpath='{range .items[*]}{"\n"}{.metadata.name}{":\t"}{range .spec.containers[*]}{.image}{", "}{end}{end}' | sort
```<commit_msg>README upate - restore/backup procedure delivered<commit_after># Overview

InfluxDB is an open source database for storing time series data. The source of time series data may come from logging and monitoring systems and IoT devices.

This is a single-instance version of InfluxDB. Multi-instance version of InfluxDB requires commercial license.

If you are interested in enterprise version of InfluxDB or you would like to learn more about InfluxDB in general, please, visit [InfluxDB website](https://www.influxdata.com/).

## About Google Click to Deploy

Popular open stacks on Kubernetes packaged by Google.

# Installation

## Quick install with Google Cloud Marketplace

Get up and running with a few clicks! Install this InfluxDB app to a
Google Kubernetes Engine cluster using Google Cloud Marketplace. Follow the
[on-screen instructions](https://console.cloud.google.com/launcher/details/google/influxdb).

## Command line instructions

### Prerequisites

#### Set up command-line tools

You'll need the following tools in your development environment:
- [gcloud](https://cloud.google.com/sdk/gcloud/)
- [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/)
- [docker](https://docs.docker.com/install/)
- [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)

#### Create a Google Kubernetes Engine cluster

Create a new cluster from the command-line.

```shell
export CLUSTER=marketplace-cluster
export ZONE=us-west1-a

gcloud container clusters create "$CLUSTER" --zone "$ZONE"
```

Configure `kubectl` to talk to the new cluster.

```shell
gcloud container clusters get-credentials "$CLUSTER" --zone "$ZONE"
```

#### Clone this repo

Clone this repo and the associated tools repo.

```shell
gcloud source repos clone google-click-to-deploy --project=k8s-marketplace-eap
gcloud source repos clone google-marketplace-k8s-app-tools --project=k8s-marketplace-eap
```

#### Install the Application resource definition

Do a one-time setup for your cluster to understand Application resource via installing Application's Custom Resource Definition.

<!--
To do that, navigate to `k8s/vendor` subdirectory of the repository and run the following command:
-->

```shell
kubectl apply -f google-marketplace-k8s-app-tools/crd/*
```

The Application resource is defined by the
[Kubernetes SIG-apps](https://github.com/kubernetes/community/tree/master/sig-apps)
community. The source code can be found on
[github.com/kubernetes-sigs/application](https://github.com/kubernetes-sigs/application).

### Install the Application

Navigate to the `influxdb` directory.

```shell
cd google-click-to-deploy/k8s/influxdb
```

#### Configure the app with environment variables

Choose the instance name and namespace for the app.

```shell
export APP_INSTANCE_NAME=influxdb-1
export NAMESPACE=default
```

Configure the container images.

```shell
export IMAGE_INFLUXDB="gcr.io/k8s-marketplace-eap/google/influxdb:latest"
```

Configure InfluxDB administrator account:

```shell
export INFLUXDB_ADMIN_USER=influxdb-admin
```

Configure password for InfluxDB administrator account (the value has to be encoded in base64)

```shell
export INFLUXDB_ADMIN_PASSWORD=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 12 | head -n 1 | tr -d '\n' | base64)
```

The images above are referenced by
[tag](https://docs.docker.com/engine/reference/commandline/tag). It is strongly
recommended to pin each image to an immutable
[content digest](https://docs.docker.com/registry/spec/api/#content-digests).
This will ensure that the installed application will always use the same images,
until you are ready to upgrade.

```shell
repo=`echo $IMAGE_INFLUXDB | cut -d: -f1`;
digest=`docker pull $IMAGE_INFLUXDB | sed -n -e 's/Digest: //p'`;
export $i="$repo@$digest";
env | grep $i;
```

#### Expand the manifest template

Use `envsubst` to expand the template. It is recommended that you save the
expanded manifest file for future updates to the application.

```shell
awk 'BEGINFILE {print "---"}{print}' manifest/* \
  | envsubst '$APP_INSTANCE_NAME $NAMESPACE $IMAGE_INFLUXDB $INFLUXDB_ADMIN_USER $INFLUXDB_ADMIN_PASSWORD' \
  > "${APP_INSTANCE_NAME}_manifest.yaml"
```

#### Apply to Kubernetes

Use `kubectl` to apply the manifest to your Kubernetes cluster.

```shell
kubectl apply -f "${APP_INSTANCE_NAME}_manifest.yaml" --namespace "${NAMESPACE}"
```

#### View the app in the Google Cloud Console

Point your browser to:

```shell
echo "https://console.cloud.google.com/kubernetes/application/${ZONE}/${CLUSTER}/${NAMESPACE}/${APP_INSTANCE_NAME}"
```

### Access InfluxDB (internally)

It is possible to connect to InfluxDB  without exposing it to public access and using `influx` tool.

Please, refer to [InfluxDB Getting Started](https://docs.influxdata.com/influxdb/v1.5/introduction/getting-started/)
for more information about `influx` usage and how to upload sample data to your InfluxDB instance.

#### Connect to InfluxDB via Pod

To do this, please identify InfluxDB's Pod using the following command:
```shell
kubectl get pods -o wide -l app.kubernetes.io/name=$APP_INSTANCE_NAME
```

Now, you can access InfluxDB using `influx` tool
```shell
kubectl exec -it "$APP_INSTANCE_NAME-influxdb-0" --namespace "$NAMESPACE" -- influx -host localhost -port 8086 -username <admin username> -password <<PASSWORD>>
```

#### Connect to InfluxDB using `kubectl port-forward` method

This method assumes that you installed `influx` tool on your local machine.
Please, refer to [InfluxDB installation instructions](https://docs.influxdata.com/influxdb/v1.5/introduction/installation/)
to learn how to do that.

You could also use a local proxy to access InfluxDB that is not exposed publicly. Run the following command in a separate background terminal:
```shell
 kubectl port-forward "${APP_INSTANCE_NAME}-influxdb-0" 8086:8086 --namespace "${NAMESPACE}"
 ```

Now, in your main terminal you can invoke `influx` tool as follows:
```shell
influx -host localhost -port 8086 -username <admin username> -password <<PASSWORD>>
```

### Access InfluxDB (externally)

This specific InfluxDB configuration was prepared to be used as internal component of your system,
e.g. as part of your log collection system consistig of Prometheus+InfluxDB+Grafana.

It is possible to expose InfluxDB to external world - it's not recommened though to do that without securing connection to the database with SSL/TLS.

In case you would like, anyway, to expose InfluxDB solution for testing purposes (for example) you can do that in the following way:

```
kubectl patch svc "$APP_INSTANCE_NAME-influxdb-svc" \
  --namespace "$NAMESPACE" \
  --patch '{"spec": {"type": "LoadBalancer"}}'
```

> **NOTE:** It might take some time for the external IP to be provisioned.

#### Extract IP addess

Get the external IP of InfluxDB instance using the following command:

#### Connect to InfluxDB via Pod

To do this, please identify InfluxDB's Pod using the following command:
```shell
kubectl get pods -o wide -l app.kubernetes.io/name=$APP_INSTANCE_NAME
```

Now, you can access InfluxDB using `influx` tool
```shell
kubectl exec -it "$APP_INSTANCE_NAME-influxdb-0" --namespace "$NAMESPACE" -- influx -host localhost -port 8086 -username <InfluxDB Admin username> -password <<PASSWORD>>
```

#### Connect to InfluxDB using `kubectl port-forward` method

This method assumes that you installed `influx` tool on your local machine. 
Please, refer to [InfluxDB installation instructions](https://docs.influxdata.com/influxdb/v1.5/introduction/installation/)
to learn how to do that.

You could also use a local proxy to access InfluxDB that is not exposed publicly. Run the following command in a separate background terminal:
```shell
 kubectl port-forward "${APP_INSTANCE_NAME}-influxdb-0" 8086:8086 --namespace "${NAMESPACE}"
 ```

Now, in your main terminal you can invoke `influx` tool as follows:
```shell
influx -host localhost -port 8086 -username <InfluxDB Admin username> -password <<PASSWORD>>
```

### Access InfluxDB (externally)

This specific InfluxDB configuration was prepared to be used as internal component of your system,
e.g. as part of your log collection system consistig of Prometheus+InfluxDB+Grafana.

It is possible to expose InfluxDB to external world - it's not recommened though to do that without securing connection to the database with SSL/TLS.

In case you would like, anyway, to expose InfluxDB solution for testing purposes (for example) you can do that in the following way:

```
kubectl patch svc "$APP_INSTANCE_NAME-influxdb-svc" \
  --namespace "$NAMESPACE" \
  --patch '{"spec": {"type": "LoadBalancer"}}'
```

> **NOTE:** It might take some time for the external IP to be provisioned.

#### Extract IP addess

Get the external IP of InfluxDB instance using the following command:

```shell
INFLUXDB_IP=$(kubectl get svc $APP_INSTANCE_NAME-influxdb-svc \
  --namespace $NAMESPACE \
  --output jsonpath='{.status.loadBalancer.ingress[0].ip}')

echo $INFLUXDB_IP
```

# Scaling

This is a single-instance version of InfluxDB. You cannot scale it.

If you are interested in multi-instance/enterprise version of InfluxDB, please, visit [InfluxDB website](https://www.influxdata.com/).

# Backup and Restore

Backup and restore of InfluxDB data will be done as it was suggested on [InfluxDB website](https://docs.influxdata.com/influxdb/v1.5/administration/backup_and_restore/)

`influxd backup` and `influxd restore` commands will be used for backing up InfluxDB databases and restoring them (respectively).

Connectivity on port 8088 to InfluxDB instance will be required to perform these operations.

Before you proceed, please, make sure that `influxdb-backup` directory on your local computer is empty.

## Backup InfluxDB data to your local computer

Navigate to the `influxdb/scripts` directory.

```shell
cd google-click-to-deploy/k8s/influxdb/scripts
```

Run `make-backup.sh` script passing the name of your InfluxDB instance as an argument to this script.
```shell
cd google-click-to-deploy/k8s/influxdb/scripts
```

Backup data will be stored in `influxdb-backup` directory on your local computer.

## Restore InfluxDB data on running InfluxDB instance
Navigate to the `influxdb/scripts` directory.

```shell
cd google-click-to-deploy/k8s/influxdb/scripts
```

Run `make-restore.sh` script passing the name of your InfluxDB instance as an argument to this script.
```shell
cd google-click-to-deploy/k8s/influxdb/scripts
```

Restore will be performed based on the data that is stored in `influxdb-backup` on your local computer.


# Upgrade

This is single-instance InfluxDB solution:
- Upgrade will case some downtime of your application
- Configuration and data of InfluxDB will be retained.

This procudure assumes that you have a new image for InfluxDB container published and being available to your Kubernetes cluster. The new image is available at <url-pointing-to-new-image>.

Start with modification of the image used for pod temaplate within InfluxDB StatefulSet:

```shell
kubectl set image statefulset "$APP_INSTANCE_NAME-influxdb" \
  influxdb=<url-pointing-to-new-image>
```

where `<url-pointing-to-new-image>` is the new image.

To check the status of Pods in the StatefulSet and the progress of deployment of new image run the following command:

```shell
kubectl get pods -l app.kubernetes.io/name=$APP_INSTANCE_NAME
```

To check the current image used by pods within `InfluxDB` K8s application, you can run the following command:

```shell
kubectl get pods -l app.kubernetes.io/name=$APP_INSTANCE_NAME -o=jsonpath='{range .items[*]}{"\n"}{.metadata.name}{":\t"}{range .spec.containers[*]}{.image}{", "}{end}{end}' | sort
```
{'fraction_non_alphanumeric': 0.1052007453837032, 'fraction_numerical': 0.005308035462194364, 'mean_word_length': 4.8256578947368425, 'pattern_counts': {'":': 6, '<': 23, '<?xml version=': 0, '>': 27, 'https://': 27, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '476246', 'n_tokens_mistral': 5563, 'n_tokens_neox': 5247, 'n_words': 1995}