<commit_before># Snap collector plugin - Ceph

This  plugin collects Ceph performance counters from the Ceph Storage System for:
* MON (Ceph Monitor Daemon)
* OSD (Ceph Object Storage Daemon)
* MDS (Ceph Metadata Server Daemon)

The perf counters data are accessed via the Ceph admin socket.
The intention is that data will be collected, aggregated and fed into another tool for graphing and analysis.

This plugin is used in the [Snap framework] (http://github.com/intelsdi-x/snap).

1. [Getting Started](#getting-started)
  * [System Requirements](#system-requirements)
  * [Installation](#installation)
  * [Configuration and Usage](#configuration-and-usage)
2. [Documentation](#documentation)
  * [Collected Metrics](#collected-metrics)
  * [Examples](#examples)
  * [Roadmap](#roadmap)
3. [Community Support](#community-support)
4. [Contributing](#contributing)
5. [License](#license)
6. [Acknowledgements](#acknowledgements)

## Getting Started

In order to use this plugin you need Ceph cluster running.

### System Requirements

* [Ceph Storage Cluster] (http://ceph.com/)
* [Ceph Administration Tool] (http://docs.ceph.com/docs/v9.0.2/man/8/ceph/)
* Root privileges might be needed

### Installation

#### To deploy the Ceph cluster

The quickest way to get a Ceph cluster up and running is to follow the Getting Started guides available at http://ceph.com/resources/downloads/. 		It can be tested also on a fake local cluster on Your machine. Read more about [how to deploy fake local Ceph cluster](VCLUSTER.md).

#### To build the plugin binary:
Fork https://github.com/intelsdi-x/snap-plugin-collector-ceph
Clone repo into `$GOPATH/src/github.com/intelsdi-x/`:

```
$ git clone https://github.com/<yourGithubID>/snap-plugin-collector-ceph.git
```

Build the plugin by running make within the cloned repo:
```
$ make
```
This builds the plugin in `/build/${GOOS}/${GOARCH}`

### Configuration and Usage
* Set up the [Snap framework](https://github.com/intelsdi-x/snap/blob/master/README.md#getting-started)
* Ensure `$SNAP_PATH` is exported
`export SNAP_PATH=$GOPATH/src/github.com/intelsdi-x/snap/linux/x86_64`

* Set proper Snap Global Config field(s) to customize Ceph's path:

Namespace | Data Type | Description
----------|-----------|-----------------------
**path** | string | Path to "ceph" executable. Defaults to"/usr/bin/ceph"
**socket_path** | string | The location of the ceph monitoring sockets. Defaults to "/var/run/ceph"
**socket_prefix** | string | The first part of socket names. Defaults to "ceph-"
**socket_ext** | string | Extension for socket filenames. Defaults to "asok"
If sockets do not have prefix, set *socket_prefix="none"*

Sample Global Config is available in folder /examples/configs.

## Documentation

To learn more about this plugin and ceph perf counters, visit:

* [ceph perf counters doc] (http://ceph.com/docs/master/dev/perf_counters)
* [Snap ceph unit test](https://github.com/intelsdi-x/snap-plugin-collector-ceph/blob/master/ceph/ceph_test.go)
* [Snap ceph examples](#examples)

Resetting the perf counters before measurement is recommended. This feature was added in the Ceph 0.90 (Hammer release):
```
$ sudo ceph daemon <daemon-name> perf reset all | <perf_cnt_name>
```

### Collected Metrics
This plugin has the ability to gather the following Ceph perf counters from:

* MON [see more...](MON_PERFCNT.md)
* MDS [see more...](MDS_PERFCNT.md)
* OSD [see more...](OSD_PERFCNT.md)

By default metrics are gathered once per second.


### Examples

Example of running Snap ceph perf counters collector and writing data to file.

Run the Snap daemon on each node with defaults settings:
```
$ $SNAP_PATH/snapd -l 1 -t 0
```
Or set custom settings in Snap Global Config in Ceph section (see examples/configs/snap-config-sample.json):
```
$ $SNAP_PATH/snapd -l 1 -t 0 --config $SNAP_CEPH_PLUGIN_DIR/examples/configs/snap-config-sample.json
```

Load ceph plugin for collecting:
```
$ $SNAP_PATH/snapctl plugin load $SNAP_CEPH_PLUGIN_DIR/build/linux/x86_64/snap-plugin-collector-ceph
Plugin loaded
Name: ceph
Version: 4
Type: collector
Signed: false
Loaded Time: Tue, 01 Dec 2015 06:19:48 EST
```
See available metrics for all ceph-daemon in cluster:
```
$ $SNAP_PATH/snapctl metric list
```

Or see available metrics only for OSDs:
```
$ $SNAP_PATH/snapctl metric list | grep ceph/osd
```
Download desired processor and publisher plugins eg.
```
$ wget http://snap.ci.snap-telemetry.io/plugins/snap-plugin-publisher-file/latest/linux/x86_64/snsap-plugin-publisher-file
```
Load file plugin for publishing:
```
$ $SNAP_PATH/snapctl plugin load snap-plugin-publisher-file
Plugin loaded
Name: file
Version: 4
Type: publisher
Signed: false
Loaded Time: Tue, 01 Dec 2015 07:45:58 EST
```

Create a task JSON file (exemplary file in examples/tasks/ceph-file.json):
```json
{
  "version": 1,
  "schedule": {
      "type": "simple",
      "interval": "1s"
  },
  "workflow": {
    "collect": {
      "metrics": {
        "/intel/storage/ceph/mon/*/cluster/num_mon": {},
        "/intel/storage/ceph/mon/*/cluster/num_osd": {},<commit_msg>Removed mentioning processor<commit_after># Snap collector plugin - Ceph

This  plugin collects Ceph performance counters from the Ceph Storage System for:
* MON (Ceph Monitor Daemon)
* OSD (Ceph Object Storage Daemon)
* MDS (Ceph Metadata Server Daemon)

The perf counters data are accessed via the Ceph admin socket.
The intention is that data will be collected, aggregated and fed into another tool for graphing and analysis.

This plugin is used in the [Snap framework] (http://github.com/intelsdi-x/snap).

1. [Getting Started](#getting-started)
  * [System Requirements](#system-requirements)
  * [Installation](#installation)
  * [Configuration and Usage](#configuration-and-usage)
2. [Documentation](#documentation)
  * [Collected Metrics](#collected-metrics)
  * [Examples](#examples)
  * [Roadmap](#roadmap)
3. [Community Support](#community-support)
4. [Contributing](#contributing)
5. [License](#license)
6. [Acknowledgements](#acknowledgements)

## Getting Started

In order to use this plugin you need Ceph cluster running.

### System Requirements

* [Ceph Storage Cluster] (http://ceph.com/)
* [Ceph Administration Tool] (http://docs.ceph.com/docs/v9.0.2/man/8/ceph/)
* Root privileges might be needed

### Installation

#### To deploy the Ceph cluster

The quickest way to get a Ceph cluster up and running is to follow the Getting Started guides available at http://ceph.com/resources/downloads/. 		It can be tested also on a fake local cluster on Your machine. Read more about [how to deploy fake local Ceph cluster](VCLUSTER.md).

#### To build the plugin binary:
Fork https://github.com/intelsdi-x/snap-plugin-collector-ceph
Clone repo into `$GOPATH/src/github.com/intelsdi-x/`:

```
$ git clone https://github.com/<yourGithubID>/snap-plugin-collector-ceph.git
```

Build the plugin by running make within the cloned repo:
```
$ make
```
This builds the plugin in `/build/${GOOS}/${GOARCH}`

### Configuration and Usage
* Set up the [Snap framework](https://github.com/intelsdi-x/snap/blob/master/README.md#getting-started)
* Ensure `$SNAP_PATH` is exported
`export SNAP_PATH=$GOPATH/src/github.com/intelsdi-x/snap/linux/x86_64`

* Set proper Snap Global Config field(s) to customize Ceph's path:

Namespace | Data Type | Description
----------|-----------|-----------------------
**path** | string | Path to "ceph" executable. Defaults to"/usr/bin/ceph"
**socket_path** | string | The location of the ceph monitoring sockets. Defaults to "/var/run/ceph"
**socket_prefix** | string | The first part of socket names. Defaults to "ceph-"
**socket_ext** | string | Extension for socket filenames. Defaults to "asok"
If sockets do not have prefix, set *socket_prefix="none"*

Sample Global Config is available in folder /examples/configs.

## Documentation

To learn more about this plugin and ceph perf counters, visit:

* [ceph perf counters doc] (http://ceph.com/docs/master/dev/perf_counters)
* [Snap ceph unit test](https://github.com/intelsdi-x/snap-plugin-collector-ceph/blob/master/ceph/ceph_test.go)
* [Snap ceph examples](#examples)

Resetting the perf counters before measurement is recommended. This feature was added in the Ceph 0.90 (Hammer release):
```
$ sudo ceph daemon <daemon-name> perf reset all | <perf_cnt_name>
```

### Collected Metrics
This plugin has the ability to gather the following Ceph perf counters from:

* MON [see more...](MON_PERFCNT.md)
* MDS [see more...](MDS_PERFCNT.md)
* OSD [see more...](OSD_PERFCNT.md)

By default metrics are gathered once per second.


### Examples

Example of running Snap ceph perf counters collector and writing data to file.

Run the Snap daemon on each node with defaults settings:
```
$ $SNAP_PATH/snapd -l 1 -t 0
```
Or set custom settings in Snap Global Config in Ceph section (see examples/configs/snap-config-sample.json):
```
$ $SNAP_PATH/snapd -l 1 -t 0 --config $SNAP_CEPH_PLUGIN_DIR/examples/configs/snap-config-sample.json
```

Load ceph plugin for collecting:
```
$ $SNAP_PATH/snapctl plugin load $SNAP_CEPH_PLUGIN_DIR/build/linux/x86_64/snap-plugin-collector-ceph
Plugin loaded
Name: ceph
Version: 4
Type: collector
Signed: false
Loaded Time: Tue, 01 Dec 2015 06:19:48 EST
```
See available metrics for all ceph-daemon in cluster:
```
$ $SNAP_PATH/snapctl metric list
```

Or see available metrics only for OSDs:
```
$ $SNAP_PATH/snapctl metric list | grep ceph/osd
```
Download desired publisher plugin eg.
```
$ wget http://snap.ci.snap-telemetry.io/plugins/snap-plugin-publisher-file/latest/linux/x86_64/snsap-plugin-publisher-file
```
Load file plugin for publishing:
```
$ $SNAP_PATH/snapctl plugin load snap-plugin-publisher-file
Plugin loaded
Name: file
Version: 4
Type: publisher
Signed: false
Loaded Time: Tue, 01 Dec 2015 07:45:58 EST
```

Create a task JSON file (exemplary file in examples/tasks/ceph-file.json):
```json
{
  "version": 1,
  "schedule": {
      "type": "simple",
      "interval": "1s"
  },
  "workflow": {
    "collect": {
      "metrics": {
        "/intel/storage/ceph/mon/*/cluster/num_mon": {},
        "/intel/storage/ceph/mon/*/cluster/num_osd": {},
{'fraction_non_alphanumeric': 0.1334120425029516, 'fraction_numerical': 0.011216056670602124, 'mean_word_length': 4.28051948051948, 'pattern_counts': {'":': 18, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1576382', 'n_tokens_mistral': 3467, 'n_tokens_neox': 3199, 'n_words': 1050}