<commit_before>mpp
===

[![Build Status](https://travis-ci.org/matt-deboer/mpp.svg?branch=master)](https://travis-ci.org/matt-deboer/mpp)
[![Docker Pulls](https://img.shields.io/docker/pulls/mattdeboer/mpp.svg)](https://hub.docker.com/r/mattdeboer/mpp/)

Multi-prometheus proxy (**mpp**) exists to forward incoming query requests to one of a set
of multiple prometheus instances deployed as HA duplicates of each other.


Motivation
---

As the recommended pattern for running Prometheus in HA mode is to [run multiple duplicate instances](https://github.com/prometheus/prometheus/issues/1500)
(same configuration, scraping the same targets independently), a method is needed to route queries
appropriately between those instances to provide a seemless experience for clients when individual
instance failures occur.

How it works
---

MPP acts as a proxy sitting in front of multiple prometheus instances, choosing one or more instances
to receive requests based on a configurable selector strategy. Candidate instances are found using
a configured _locator_ mechanism, of which Marathon, Kubernetes and endpoints file are supported.

Discovery
---

Prometheus endpoints can be discovered via the Marathon API, the Kubernetes API, or by providing an 
endpoints file which is scanned on a regular interval.

**Marathon** discovery is configured using:

- `--marathon-url`: The marathon API endpoint to contact.
- `--marathon-apps`: A comma-separated list of apps to query for endpoints.
- `--marathon-principal-secret`: (Optional) A DCOS principal-secret object used to authenticate to Marathon.
- `--marathon-auth-endpoint`: (Optional) Overrides the auth-endpoint contained within the principal secret object.

**Kubernetes** discovery _(Coming soon)_ is configured using:

- `--kubeconfig`: The path to the kubeconfig file used to locate the cluster and authenticate.
- `--kube-pod-selector`: A pod-selector string used to locate the pods containing the endpoints.  

**Endpoints file** discovery is configured using `--endpoints-file` -- the path to a file containing one
endpoint per line.

Selection
---

Traffic is routed based on the chosen `--routing-strategy`:

- `single-most-data`: This strategy always routes traffic to a single prometheus endpoint, determined
  by whichever endpoint contains the _most_ data, measured by total metrics count.

- `minimum-history-sticky`: This strategy routes traffic to a randomly selected prometheus endpoint having<commit_msg>update readme to reflect latest features<commit_after>mpp
===

[![Build Status](https://travis-ci.org/matt-deboer/mpp.svg?branch=master)](https://travis-ci.org/matt-deboer/mpp)
[![Docker Pulls](https://img.shields.io/docker/pulls/mattdeboer/mpp.svg)](https://hub.docker.com/r/mattdeboer/mpp/)

Multi-prometheus proxy (**mpp**) exists to forward incoming query requests to one of a set
of multiple prometheus instances deployed as HA duplicates of each other.


Motivation
---

As the recommended pattern for running Prometheus in HA mode is to [run multiple duplicate instances](https://github.com/prometheus/prometheus/issues/1500)
(same configuration, scraping the same targets independently), a method is needed to route queries
appropriately between those instances to provide a seemless experience for clients when individual
instance failures occur.

How it works
---

MPP acts as a proxy sitting in front of multiple prometheus instances, choosing one or more instances
to receive requests based on a configurable selector strategy. Candidate instances are found using
a configured _locator_ mechanism, of which Marathon, Kubernetes and endpoints file are supported.

Discovery
---

Prometheus endpoints can be discovered via the Marathon API, the Kubernetes API, or by providing an 
endpoints file which is scanned on a regular interval.

**Marathon** discovery is configured using:

- `--marathon-url`: The marathon API endpoint to contact.
- `--marathon-apps`: A comma-separated list of apps to query for endpoints.
- `--marathon-principal-secret`: (Optional) A DCOS principal-secret object used to authenticate to Marathon.
- `--marathon-auth-endpoint`: (Optional) Overrides the auth-endpoint contained within the principal secret object.

**Kubernetes** discovery _(Coming soon)_ is configured using:

- `--kubeconfig`: The path to the kubeconfig file used to locate the cluster and authenticate.
- `--kube-pod-selector`: A pod-selector string used to locate the pods containing the endpoints.  

**Endpoints file** discovery is configured using `--endpoints-file` -- the path to a file containing one
endpoint per line.

Selection
---

Traffic is routed based on the chosen `--routing-strategy`:

- `single-most-data`: This strategy always routes traffic to a single prometheus endpoint, determined
  by whichever endpoint contains the _most_ data, measured by total metrics count.

- `minimum-history-sticky:{min-history-duration}`: This strategy routes traffic to a randomly selected prometheus endpoint having
{'fraction_non_alphanumeric': 0.085, 'fraction_numerical': 0.0016, 'mean_word_length': 4.708904109589041, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '997213', 'n_tokens_mistral': 1377, 'n_tokens_neox': 1279, 'n_words': 582}