<commit_before># Scope HTTP Statistics Plugin

The Scope HTTP Statistics plugin provides HTTP traffic statistics to the [Weave Scope](https://github.com/weaveworks/scope) user using [eBPF](http://man7.org/linux/man-pages/man2/bpf.2.html).
The HTTP Statistics plugin does not use any application-level instrumentation, it has a negligible performance toll (metrics are obtained in-kernel without any packet copying to user space).

**Note:** This plugin needs a [recent kernel version with eBPF support](https://github.com/iovisor/bcc/blob/master/INSTALL.md#kernel-configuration).
It will not compile on current [dlite](https://github.com/nlf/dlite) and boot2docker hosts.
You need `/sys/kernel/debug/` in order to be able to build the eBPF program generated by the plugin.

<img src="imgs/http-statistics.png" width="800" alt="Scope HTTP Statistics plugin screenshot" align="center">

## How to Run Scope HTTP Statistics Plugin

This plugin requires:

* kernel version [>=4.2](https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md)
 * Attaching to kprobes requires >=4.1.
 * `bpf_probe_read()` requires >=4.1.
 * `bpf_get_current_pid_tgid()` requires >=4.2.
* Kernel build directory to be available in `/lib/modules/<kernel-version>/build`. Depending on your distribution you might need to add this symlink: `ln -s /lib/modules/<kernel-version>/build /lib/modules/<kernel-version>/source`.

The Scope HTTP Statistics plugin can be executed stand alone.
It will respond to `GET /report` request on the `/var/run/scope/plugins/http-statistics/http-statistics.sock` in a JSON format.
If the running plugin has been registered by Scope, you will see it in the list of `PLUGINS` in the bottom right of the UI (see the green rectangle in the above figure).
The measured values are shown in the *STATUS* section (see the circle in the above figure).

### Using a pre-built Docker image

If you want to make sure of running the latest available version of the plugin, you can pull the image from docker hub.

```
docker pull weaveworksplugins/scope-http-statistics:latest
```

To run the Scope HTTP Statistics plugin you just need to run the following command.

```
sudo docker run --rm -it \
	  --privileged --net=host --pid=host \
	  -v /lib/modules:/lib/modules \
	  -v /usr/src:/usr/src \
	  -v /sys/kernel/debug/:/sys/kernel/debug/ \
	  -v /var/run/scope/plugins:/var/run/scope/plugins \
	  --name weaveworksplugins-scope-http-statistics weaveworksplugins/scope-http-statistics
```

### Kubernetes

If you want to use the Scope HTTP Statistics plugin in an already set up Kubernetes cluster with Weave Scope running on it, you just need to run:

```
kubectl create -f https://github.com/weaveworks-plugins/scope-http-statistics/tree/master/deployments/k8s-http-statistics.yaml
```

### Recompiling
* Run the HTTP Statistics plugin
	* `git clone <EMAIL>.com:weaveworks-plugins/scope-http-statistics.git`
	* `cd scope-http-statistics; make`

## Testing

* Run an `nginx` instance `docker run --rm --name http-statistics-nginx -p 8080:80 nginx`
* Run `sh test-http-statistics.sh`, press Ctrl+c to terminate the test.
* Go to the Weave Scope UI [http://localhost:4040](http://localhost:4040).
* Open the `http-statistics-nginx` container.
* Click on `nginx: worker process` in the "Processes" table.

## Metrics
The HTTP Statistics plugin currently implements the following metrics:<commit_msg>Fixing documentation.<commit_after># Scope HTTP Statistics Plugin

The Scope HTTP Statistics plugin provides HTTP traffic statistics to the [Weave Scope](https://github.com/weaveworks/scope) user using [eBPF](http://man7.org/linux/man-pages/man2/bpf.2.html).
The HTTP Statistics plugin does not use any application-level instrumentation, it has a negligible performance toll (metrics are obtained in-kernel without any packet copying to user space).

**Note:** This plugin needs a [recent kernel version with eBPF support](https://github.com/iovisor/bcc/blob/master/INSTALL.md#kernel-configuration).
It will not compile on current [dlite](https://github.com/nlf/dlite) and boot2docker hosts.
You need `/sys/kernel/debug/` in order to be able to build the eBPF program generated by the plugin.

<img src="imgs/http-statistics.png" width="800" alt="Scope HTTP Statistics plugin screenshot" align="center">

## How to Run Scope HTTP Statistics Plugin

This plugin requires:

* kernel version [>=4.2](https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md)
 * Attaching to kprobes requires >=4.1.
 * `bpf_probe_read()` requires >=4.1.
 * `bpf_get_current_pid_tgid()` requires >=4.2.
* Kernel build directory to be available in `/lib/modules/<kernel-version>/build`. Depending on your distribution you might need to add this symlink: `ln -s /lib/modules/<kernel-version>/build /lib/modules/<kernel-version>/source`.

The Scope HTTP Statistics plugin can be executed stand alone.
It will respond to `GET /report` request on the `/var/run/scope/plugins/http-statistics/http-statistics.sock` in a JSON format.
If the running plugin has been registered by Scope, you will see it in the list of `PLUGINS` in the bottom right of the UI (see the green rectangle in the above figure).
The measured values are shown in the *STATUS* section (see the circle in the above figure).

### Using a pre-built Docker image

If you want to make sure of running the latest available version of the plugin, you can pull the image from docker hub.

```
docker pull weaveworksplugins/scope-http-statistics:latest
```

To run the Scope HTTP Statistics plugin you just need to run the following command.

```
sudo docker run --rm -it \
	  --privileged --net=host --pid=host \
	  -v /lib/modules:/lib/modules \
	  -v /usr/src:/usr/src \
	  -v /sys/kernel/debug/:/sys/kernel/debug/ \
	  -v /var/run/scope/plugins:/var/run/scope/plugins \
	  --name weaveworksplugins-scope-http-statistics weaveworksplugins/scope-http-statistics
```

### Kubernetes

If you want to use the Scope HTTP Statistics plugin in an already set up Kubernetes cluster with Weave Scope running on it, you just need to run:

```
kubectl create -f https://raw.githubusercontent.com/weaveworks-plugins/scope-http-statistics/master/deployments/k8s-http-statistics.yaml
```

### Recompiling
* Run the HTTP Statistics plugin
	* `git clone <EMAIL>:weaveworks-plugins/scope-http-statistics.git`
	* `cd scope-http-statistics; make`

## Testing

* Run an `nginx` instance `docker run --rm --name http-statistics-nginx -p 8080:80 nginx`
* Run `sh test-http-statistics.sh`, press Ctrl+c to terminate the test.
* Go to the Weave Scope UI [http://localhost:4040](http://localhost:4040).
* Open the `http-statistics-nginx` container.
* Click on `nginx: worker process` in the "Processes" table.

## Metrics
The HTTP Statistics plugin currently implements the following metrics:
{'fraction_non_alphanumeric': 0.11153507484590548, 'fraction_numerical': 0.008805400645729381, 'mean_word_length': 4.282945736434108, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 21, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '995426', 'n_tokens_mistral': 2117, 'n_tokens_neox': 2006, 'n_words': 754}