<commit_before># carbon-clickhouse
Graphite metrics receiver with ClickHouse as storage

## Production status
Last releases are stable and ready for production use

## TL;DR
[Preconfigured docker-compose](https://github.com/lomik/graphite-clickhouse-tldr)

## Build
```sh
# build binary
git clone https://github.com/lomik/carbon-clickhouse.git
cd carbon-clickhouse
make
```

## ClickHouse configuration

1. Add `graphite_rollup` section to config.xml. Sample [here](https://clickhouse.yandex/docs/en/operations/table_engines/graphitemergetree/). You can use [carbon-schema-to-clickhouse](https://github.com/bzed/carbon-schema-to-clickhouse) for generate rollup xml from graphite [storage-schemas.conf](http://graphite.readthedocs.io/en/latest/config-carbon.html#storage-schemas-conf).

2. Create tables.
Be aware that in these examples the tables are partitioned by day.
If you want old Clickhouse behaviour with monthly partitions then change toYYYYMMDD to toYYYYMM everywhere.

```sql
CREATE TABLE graphite ( 
  Path String,  
  Value Float64,  
  Time UInt32,  
  Date Date,  
  Timestamp UInt32
) ENGINE = GraphiteMergeTree('graphite_rollup')
PARTITION BY toYYYYMMDD(Date)
ORDER BY (Path, Time);

-- optional table for faster metric search
CREATE TABLE graphite_tree (
  Level UInt32,
  Path String,
  Version UInt32
) ENGINE = ReplacingMergeTree(Version)
PARTITION BY (Level)
ORDER BY (Level, Path);

-- optional table for daily series (see config file)
CREATE TABLE graphite_series (
  Date Date,
  Level UInt32,
  Path String,
  Version UInt32
) ENGINE = ReplacingMergeTree(Version)
PARTITION BY toYYYYMMDD(Date)
ORDER BY (Level, Path, Date);

-- optional table for storing Graphite tags
CREATE TABLE graphite_tagged (
  Date Date,
  Tag1 String,<commit_msg>partition of graphite_tree is not required<commit_after># carbon-clickhouse
Graphite metrics receiver with ClickHouse as storage

## Production status
Last releases are stable and ready for production use

## TL;DR
[Preconfigured docker-compose](https://github.com/lomik/graphite-clickhouse-tldr)

## Build
```sh
# build binary
git clone https://github.com/lomik/carbon-clickhouse.git
cd carbon-clickhouse
make
```

## ClickHouse configuration

1. Add `graphite_rollup` section to config.xml. Sample [here](https://clickhouse.yandex/docs/en/operations/table_engines/graphitemergetree/). You can use [carbon-schema-to-clickhouse](https://github.com/bzed/carbon-schema-to-clickhouse) for generate rollup xml from graphite [storage-schemas.conf](http://graphite.readthedocs.io/en/latest/config-carbon.html#storage-schemas-conf).

2. Create tables.
Be aware that in these examples the tables are partitioned by day.
If you want old Clickhouse behaviour with monthly partitions then change toYYYYMMDD to toYYYYMM everywhere.

```sql
CREATE TABLE graphite ( 
  Path String,  
  Value Float64,  
  Time UInt32,  
  Date Date,  
  Timestamp UInt32
) ENGINE = GraphiteMergeTree('graphite_rollup')
PARTITION BY toYYYYMMDD(Date)
ORDER BY (Path, Time);

-- optional table for faster metric search
CREATE TABLE graphite_tree (
  Level UInt32,
  Path String,
  Version UInt32
) ENGINE = ReplacingMergeTree(Version)
ORDER BY (Level, Path);

-- optional table for daily series (see config file)
CREATE TABLE graphite_series (
  Date Date,
  Level UInt32,
  Path String,
  Version UInt32
) ENGINE = ReplacingMergeTree(Version)
PARTITION BY toYYYYMMDD(Date)
ORDER BY (Level, Path, Date);

-- optional table for storing Graphite tags
CREATE TABLE graphite_tagged (
  Date Date,
  Tag1 String,
{'fraction_non_alphanumeric': 0.09101187111362352, 'fraction_numerical': 0.009609949123798756, 'mean_word_length': 4.811165845648604, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2381422', 'n_tokens_mistral': 1144, 'n_tokens_neox': 1072, 'n_words': 347}