<commit_before># influxdb-grafana-docker
Docker compose definition file with `InfluxDB 1.1.1` and `Grafana 4.0.1` plus `bash script` for creation of sample data.

## First time setup
### run docker containers
```
sudo docker-compose up -d
```
### create database
```
curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb"
```
### configure urls which will be monitored via curl
```
cp conf/services.list.dist conf/services.list
vim conf/services.list
```
### start bash script to collect data
```
./bin/collect_data_loop # it will run in foreground (and block the terminal)
```
### check if data are there
```
curl -G 'http://localhost:8086/query' --data-urlencode "db=mydb" --data-urlencode "q=SELECT * FROM http_responses"
```
### Open Grafana and setup Data sources, Dashboard
- <http://localhost:3000> - admin/admin
- TODO write down how to setup sources


## Basic Operate
<commit_msg>Add Grafana setup to README<commit_after># influxdb-grafana-docker
Docker compose definition file with `InfluxDB 1.1.1` and `Grafana 4.0.1` plus `bash script` for creation of sample data.

## First time setup
### run docker containers
```
sudo docker-compose up -d
```
### create database
```
curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb"
```
### configure urls which will be monitored via curl
```
cp conf/services.list.dist conf/services.list
vim conf/services.list
```
### start bash script to collect data
```
./bin/collect_data_loop # it will run in foreground (and block the terminal)
```
### check if data are there
```
curl -G 'http://localhost:8086/query' --data-urlencode "db=mydb" --data-urlencode "q=SELECT * FROM http_responses"
```
### Open Grafana and setup Data sources, Dashboard
- <http://localhost:3000> - admin/admin
#### Data source
- Name: not important
- Default: checked
- Type: InfluxDB
- Url: http://influxdb:8086 (we can use container name thanks to the docker link)
- Access: Proxy
- Database: mydb (the name which is in the `collect_data` bash script)
- User,Password: <PASSWORD>
#### Dashboard
- New dashboard: (top left "Home" -> Create New -> Graph)
- Click "Panel Title" -> Edit 
- On the Metrics tab setup the query `A`
 - `select measurement`: "http_responses"
 - WHERE name = ..select tag value..
 - `field(value)` = http_code (for example)
 - ALIAS BY: .. select your name
 - Group by time interval: >5s


## Basic Operate

{'fraction_non_alphanumeric': 0.13931376601901613, 'fraction_numerical': 0.01694915254237288, 'mean_word_length': 4.148936170212766, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5021719', 'n_tokens_mistral': 824, 'n_tokens_neox': 747, 'n_words': 278}