<commit_before># Lustre Metrics Exporter

Prometheus exporter for Lustre metrics.

## Building and running

TODO

### Flags

TODO

### What's exported?

Design plans

1. Export all proc data from all nodes running the Lustre Exporter that can function as a counter type (will save histogram-type work for later).
2. Identify redundant data (if it exists).
- Deduplication would be done, at first, by enabling flags to identify the node type with a configuration flag.
3. Add in:
- Histogram data
- Other data sources (CLI data that isn't present in /proc, for example). Users will be able to disable non-proc sources via a configuration flag.<commit_msg>README.md: Update with build and run instructions<commit_after># Lustre Metrics Exporter

Prometheus exporter for Lustre metrics.

## Getting

'go get github.com/joehandzik/lustre_exporter'

## Building

'cd $GOPATH/src/github.com/joehandzik/lustre_exporter'
'make'

## Running

./lustre_exporter

### Flags

TODO, but the current plan is to create flags that would define what node metrics are pulled from a running instance (e.g., OSS and MDS metrics would each have their own flag to disable). Also, we'd have flags to disable non-procfs metrics.

### What's exported?

Design plans

1. Export all proc data from all nodes running the Lustre Exporter that can function as a counter type (will save histogram-type work for later). 
-STATUS: In progress
2. Identify redundant data (if it exists).
- Deduplication would be done, at first, by enabling flags to identify the node type with a configuration flag.
-STATUS: In progress
3. Add in:
- Histogram data
- Other data sources (CLI data that isn't present in /proc, for example). Users will be able to disable non-proc sources via a configuration flag.
-STATUS: Not yet started
{'fraction_non_alphanumeric': 0.06726457399103139, 'fraction_numerical': 0.0033632286995515697, 'mean_word_length': 4.219298245614035, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4341424', 'n_tokens_mistral': 520, 'n_tokens_neox': 501, 'n_words': 258}