<commit_before># Lustre Metrics Exporter

Prometheus exporter for Lustre metrics.

## Getting

go get github.com/HewlettPackard/lustre_exporter

## Building

cd $GOPATH/src/github.com/HewlettPackard/lustre_exporter

make

## Running

./lustre_exporter

### Flags

TODO, but the current plan is to create flags that would define what node metrics are pulled from a running instance (e.g., OSS and MDS metrics would each have their own flag to disable). Also, we'd have flags to disable non-procfs metrics.

### What's exported?

Design plans

1. Export all proc data from all nodes running the Lustre Exporter that can function as a counter type (will save histogram-type work for later).
  - STATUS: Mostly complete as far as we can tell. We still want to parse rpcstats and jobstats at minimum before calling this complete.
2. Identify redundant data (if it exists).
  - Deduplication would be done, at first, by enabling flags to identify the node type with a configuration flag.
  - STATUS: No problems so far, saving for future work.
3. Add in:
  - Histogram data
    - STATUS: We have some of this via the 'brw_stats' file. We have created the histograms within Grafana in early tests.
  - Other data sources (CLI data that isn't present in /proc, for example). Users will be able to disable non-proc sources via a configuration flag.
    - STATUS: Not yet started
    
### Contributing

To contribute to this HPE project, you'll need to fill out a CLA (Contributor License Agreement). If you would like to contribute anything more than a bug fix (feature, architectural change, etc), please file an issue and we'll get in touch with you to have you fill out the CLA. 
<commit_msg>Increase header size for Contributing section<commit_after># Lustre Metrics Exporter

Prometheus exporter for Lustre metrics.

## Getting

go get github.com/HewlettPackard/lustre_exporter

## Building

cd $GOPATH/src/github.com/HewlettPackard/lustre_exporter

make

## Running

./lustre_exporter

### Flags

TODO, but the current plan is to create flags that would define what node metrics are pulled from a running instance (e.g., OSS and MDS metrics would each have their own flag to disable). Also, we'd have flags to disable non-procfs metrics.

### What's exported?

Design plans

1. Export all proc data from all nodes running the Lustre Exporter that can function as a counter type (will save histogram-type work for later).
  - STATUS: Mostly complete as far as we can tell. We still want to parse rpcstats and jobstats at minimum before calling this complete.
2. Identify redundant data (if it exists).
  - Deduplication would be done, at first, by enabling flags to identify the node type with a configuration flag.
  - STATUS: No problems so far, saving for future work.
3. Add in:
  - Histogram data
    - STATUS: We have some of this via the 'brw_stats' file. We have created the histograms within Grafana in early tests.
  - Other data sources (CLI data that isn't present in /proc, for example). Users will be able to disable non-proc sources via a configuration flag.
    - STATUS: Not yet started
    
## Contributing

To contribute to this HPE project, you'll need to fill out a CLA (Contributor License Agreement). If you would like to contribute anything more than a bug fix (feature, architectural change, etc), please file an issue and we'll get in touch with you to have you fill out the CLA. 

{'fraction_non_alphanumeric': 0.05671466353217749, 'fraction_numerical': 0.001763150161622098, 'mean_word_length': 3.9048991354466858, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6213791', 'n_tokens_mistral': 961, 'n_tokens_neox': 927, 'n_words': 509}